-- Logs begin at Wed 2022-05-25 13:53:10 UTC, end at Wed 2022-05-25 15:30:22 UTC. -- May 25 14:34:00.311027 dv-cinder-830283-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 14:34:00.320225 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 14:34:00 2022] *** May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: nodename: dv-cinder-830283-3-iscsi May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: machine: x86_64 May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: clock source: unix May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: pcre jit disabled May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: detected number of CPU cores: 4 May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: current working directory: / May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: detected binary path: /usr/bin/uwsgi-core May 25 14:34:00.322213 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: your processes number limit is 47680 May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: your memory page size is 4096 bytes May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: detected max file descriptor number: 2048 May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: lock engine: pthread robust mutexes May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: thunder lock: enabled May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 14:34:00.323012 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 14:34:00.366906 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: Python main interpreter initialized at 0x556f7fdec9c0 May 25 14:34:00.366906 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: python threads support enabled May 25 14:34:00.366906 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: your server socket listen backlog is limited to 100 connections May 25 14:34:00.366906 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: your mercy for graceful operations on workers is 90 seconds May 25 14:34:00.367240 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: mapped 403077 bytes (393 KB) for 2 cores May 25 14:34:00.367314 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: *** Operational MODE: preforking *** May 25 14:34:00.367590 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: *** uWSGI is running in multiple interpreter mode *** May 25 14:34:00.367590 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: spawned uWSGI master process (pid: 92290) May 25 14:34:00.367788 dv-cinder-830283-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 14:34:00.369220 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: spawned uWSGI worker 1 (pid: 92305, cores: 1) May 25 14:34:00.369220 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: spawned uWSGI worker 2 (pid: 92306, cores: 1) May 25 14:34:00.369370 dv-cinder-830283-3-iscsi devstack@keystone.service[92290]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 14:34:06.042403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:34:06.042403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: from cryptography.utils import int_from_bytes May 25 14:34:06.043221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:34:06.043221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: from cryptography.utils import int_from_bytes May 25 14:34:06.043632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:34:06.043632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: from cryptography.utils import int_from_bytes May 25 14:34:06.043750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:34:06.043750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: from cryptography.utils import int_from_bytes May 25 14:34:06.091833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:34:06.091991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:34:06.092075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:34:06.092163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:34:06.092245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] command line args: [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:34:06.092347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] command line args: [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:34:06.092459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:34:06.092614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:34:06.092699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ================================================================================ {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:34:06.092779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ================================================================================ {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:34:06.092881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] admin_token = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] admin_token = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] config_dir = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] config_dir = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] config_source = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] debug = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] config_source = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] debug = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.093899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.094899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_config_append = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_config_append = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_dir = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_dir = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.095935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_file = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_file = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.096966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.097933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.098976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.099162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.099241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] publish_errors = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] publish_errors = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.100892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] standard_threads = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] standard_threads = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.101968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] use_journal = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] use_journal = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] use_json = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] use_json = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] use_stderr = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] use_stderr = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.102998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] use_syslog = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.103074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] use_syslog = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.103224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.103300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:34:06.103476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.103561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.103713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.103796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.103966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.104998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.105969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106437 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.106941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.107904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.108890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.109985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.110940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.111898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.112917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.113970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.114990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.115148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.115229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.115452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.116914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.117973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.118839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.119971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.120955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.121972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.122938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.123871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.124966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.125976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.126920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.127855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.external = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.128959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.external = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.password = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.129926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] auth.token = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.password = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] auth.token = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.130957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.131910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.132874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.133840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.134868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.135999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.136937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.137977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.138944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.139885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.140975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.141978 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.142976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.143915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.144994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.145945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.146936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.147943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148813 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.148891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.149988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.150942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.151908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.152993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.153998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.154999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.155957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.156883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.157979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.158932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.159942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.160951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.161928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.162038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.162692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.162810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.162940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] role.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] role.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] role.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] role.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.163927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.164989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.165978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.166906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.167847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.168981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.169994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.170982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.171906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.172864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.173962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.174939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.175927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.connection = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.connection = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.176972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.177972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.178622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.178761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.178889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.178967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.179921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.180941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:34:06.181021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:34:06.181157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:34:06.211271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.211404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.211754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.211845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.211987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.212086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.220515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.220601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.249318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.249492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.249593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.249732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.302112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.302454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.354519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.354776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.355128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.355366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.420945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.421800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.444939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.445720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.456705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.456960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.469821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.470994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.471128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.471213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.471342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.471556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:34:06.490496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.490730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:34:06.490831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.490967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:34:06.491075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.491281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:34:06.491361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.491483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:34:06.491564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.491781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.491870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.492034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.492114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.492289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.492369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.492540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.503275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.503495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.503590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.503773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.503855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.504039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.504121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.504316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.512178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.512392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.515463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.515682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.515764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.515960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.516859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.517940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.518019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.518207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.518289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.518471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.545522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.545785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.547276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.547525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.547609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.547804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.552859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.553064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.553149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.553340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.553422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.553619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.560633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.560847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.560931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.561127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.565357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.565555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.565763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.565953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.566895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.567084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.567164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.567349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.567432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.567610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.586439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.586903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.587001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.587178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.591646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.591826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.595316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.595487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.595610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.595754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.601189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.601365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.604808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.604963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.608607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.608729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.608905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.609883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.610043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.610156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.629112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.629282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.629921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.637855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.637940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.639619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.639709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.639891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.639974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.642945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.643032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.643225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.643318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.643503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.643585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.643778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.643859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.644071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.644154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.657893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.658168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.658250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.658444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.658527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.658757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.658841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.659895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.678025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.678436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.681272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.681715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.684581 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.685031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.686168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.686651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.687822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.688108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.688287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.688370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.688579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.688863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.696253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.696779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.697860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.698399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.701213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.701507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.701705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.701861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.701999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.702091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.702283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.702562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.711871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.712156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.712489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.712572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.712776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.713161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.720336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.720613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.720889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:34:06.721155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.721982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.722822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.723883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.724075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.724362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.724607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:34:06.762688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f7fdec9c0 pid: 92306 (default app) May 25 14:34:06.763527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f7fdec9c0 pid: 92305 (default app) May 25 14:34:06.765863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7c04-9a2e-4969-92d2-dee3151ee633 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:06.766156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7c04-9a2e-4969-92d2-dee3151ee633 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:06.766433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7c04-9a2e-4969-92d2-dee3151ee633 None None] PATH_INFO: `/v3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:06.770718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1/1] 10.222.0.74 () {50 vars in 791 bytes} [Wed May 25 14:34:06 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:15.626601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:15.627151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:15.627151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:15.631995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1/2] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:15.637127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:15.637555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:15.637684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:15.706293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: INFO dbcounter [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] Registered counter for database keystone May 25 14:34:15.727747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG oslo_db.sqlalchemy.engines [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92306) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:34:15.733861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writer thread running {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:34:15.911891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG passlib.handlers.bcrypt [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92306) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:34:15.911891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG passlib.handlers.bcrypt [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92306) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:34:15.964224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:15.965794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:16.095047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 2/3] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 25 14:34:16.256529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:16.326174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: INFO dbcounter [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] Registered counter for database keystone May 25 14:34:16.346187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG oslo_db.sqlalchemy.engines [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92305) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:34:16.353046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writer thread running {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:34:16.475609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87455e5-6e77-4e76-a2d1-4f511a39babd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:16.479915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:16.480577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.480865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.481147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.489162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:16.540094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:34:16.540387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f33b18-ff62-4519-80d9-340ff24c4f05 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:16.549754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 2/4] 10.222.0.74 () {58 vars in 1324 bytes} [Wed May 25 14:34:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:34:16.555922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:16.570603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9fd71-21cd-49b2-b67f-797a2df8bc79 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:16.574996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:16.576140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.576427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.576705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] PATH_INFO: `/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.578209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] RBAC: Authorizing `identity:get_project(project_id=77e7e794d9844bf98e02bc9b4f6dae9f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:16.736420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:34:16.737071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3188a-6ac2-425f-a57c-12840a8927fc admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:16.741987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 3/5] 10.222.0.74 () {56 vars in 1207 bytes} [Wed May 25 14:34:16 2022] GET /identity/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:18.899117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28e37-b13e-48e6-a219-1d69e460fc7c None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:18.904267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28e37-b13e-48e6-a219-1d69e460fc7c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:18.904267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28e37-b13e-48e6-a219-1d69e460fc7c None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:18.904267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 3/6] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:18.909119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:18.909489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:18.909692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:18.929437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:18.931002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:18.938907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 4/7] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 14:34:18.960767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28e37-b13e-48e6-a219-1d69e460fc7c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:18.973111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28e37-b13e-48e6-a219-1d69e460fc7c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:18.977781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:18.978398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:18.978721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:18.979001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:18.986588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:18.988361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6177d-4464-42b4-8903-96db998247df admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:18.997486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 4/8] 10.222.0.74 () {58 vars in 1324 bytes} [Wed May 25 14:34:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:34:19.002522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:19.011995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d35d51-9da1-4a6b-b5b7-f18e5ce65363 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:19.016389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:19.017240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:19.017531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:19.017811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:19.019246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:19.021072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf545f6-8c76-49e3-a108-7238c28b7258 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:19.023258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 5/9] 10.222.0.74 () {56 vars in 1198 bytes} [Wed May 25 14:34:19 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:21.255028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f95414-7d5d-431f-87ae-63c8c66b5bdc None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.255580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f95414-7d5d-431f-87ae-63c8c66b5bdc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.255580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f95414-7d5d-431f-87ae-63c8c66b5bdc None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.256951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 5/10] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:21.262151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.262541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.262755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.270296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e415f0-3c2a-49d7-98b5-51b54075ede5 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.270587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e415f0-3c2a-49d7-98b5-51b54075ede5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.270898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e415f0-3c2a-49d7-98b5-51b54075ede5 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.272720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 6/11] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:21.277654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.277948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.278232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.279749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:21.280938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:21.318296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG passlib.handlers.bcrypt [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92305) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:34:21.318780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG passlib.handlers.bcrypt [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92305) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:34:21.359996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 6/12] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:21.366168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:21.367810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:21.382249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.406207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:21.468487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 7/13] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:21.486464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ce2a6-6ced-4649-900b-5085edc35d62 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.490530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.519100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.519832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.520120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.520402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.532756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:21.544981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:21.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:21.546905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 7/14] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:21.552344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.637252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:21.655340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8c48-f5bc-4fe9-8026-c2d07e6a0c82 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.688669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.689335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.689624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.689907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.690681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:21.691000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:21.692407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:21.696959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ea9ba-c046-48db-b01a-83b0d744fd62 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.699242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:21.704479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 8/15] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:21.710552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.736276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.737183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.737478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.737816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.804457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:21.805437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80d67-b7d0-4011-b261-610c3d32e98f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.805896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:21.838787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.839645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.839931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.840208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.841445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:21.842909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:21.844500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 9/16] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:21.849892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.864218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:21.902802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 8/17] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:34:21.908364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.911623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:21.948164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd16e-0d1d-4238-93fb-bd07b97360c1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.981776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.982560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.982874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.983148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.984418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:21.985867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:21.987522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 10/18] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:21.993035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:22.008003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a20c-f96d-48cc-94c6-7cb484b3c9ae None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:22.040524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:22.041310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:22.041595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:22.041876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:22.061589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:22.063019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:22.076481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:34:22.084252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:22.086766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:22.087741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:34:22.087741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:34:22.087741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:34:22.087741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:22.087741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 9/19] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:22.091583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:22.099794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610e596-936a-41f4-b305-cbc39f3fe8a9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:22.135611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:22.136496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:22.136782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:22.137059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:22.137892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.domain.id=default, user.id=e695952cd3d8473291efec1acc9a5e16) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:22.138207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:22.139696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baa2a0-1ea0-4137-ab22-4eaafc169f69 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:22.146691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 11/20] 10.222.0.74 () {56 vars in 1322 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/role_assignments?user.id=e695952cd3d8473291efec1acc9a5e16&scope.domain.id=default&role.id=d38d6cda16454360a4f48025ab4df728 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:22.148138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:22.194794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ae00d-5931-4e8d-8ef9-8a68d5fe84a8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:22.230858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:22.231527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:22.231809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:22.232087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:22.232827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:22.233142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:22.234566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70cf72-67ef-465b-a952-c929f4e09d52 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:22.254183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 10/21] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:24.895058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7870c-953a-4547-b72e-3cb97f3f7d2d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:24.896088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7870c-953a-4547-b72e-3cb97f3f7d2d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:24.896632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7870c-953a-4547-b72e-3cb97f3f7d2d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:24.898230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 12/22] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:24.903887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c31de-df3c-4441-a81d-61d590b329e0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:24.913687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c31de-df3c-4441-a81d-61d590b329e0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:24.913687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c31de-df3c-4441-a81d-61d590b329e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:24.930761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a13c31de-df3c-4441-a81d-61d590b329e0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:24.932035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a13c31de-df3c-4441-a81d-61d590b329e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:25.021983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 11/23] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:25.025557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372413-825e-4208-aaa6-d32e077d7a9e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.025852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372413-825e-4208-aaa6-d32e077d7a9e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.026139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372413-825e-4208-aaa6-d32e077d7a9e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.027507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 13/24] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:25.032732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bb4c-c2c7-4610-96f7-2a0f16307c2f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.033030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bb4c-c2c7-4610-96f7-2a0f16307c2f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.033316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bb4c-c2c7-4610-96f7-2a0f16307c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.053392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f372413-825e-4208-aaa6-d32e077d7a9e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.070517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-eaa5bb4c-c2c7-4610-96f7-2a0f16307c2f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:25.073757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-eaa5bb4c-c2c7-4610-96f7-2a0f16307c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:25.164289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:25.189877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 12/25] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:25.191558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed0f-bb5d-4061-adc6-7d63ffff13e1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.192062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed0f-bb5d-4061-adc6-7d63ffff13e1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.192525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed0f-bb5d-4061-adc6-7d63ffff13e1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.194015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 13/26] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:25.195389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e10b9-e17d-4dba-b711-184253e1bfb1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.195862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e10b9-e17d-4dba-b711-184253e1bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.197323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e10b9-e17d-4dba-b711-184253e1bfb1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.199175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 14/27] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:25.204267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.204885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.205380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.221075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f372413-825e-4208-aaa6-d32e077d7a9e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:25.223379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:25.225378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:25.263906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:25.264706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.264989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.265264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.279259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d0234611-b46a-4094-a77f-c4a19986deef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:25.289737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:25.290566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:25.291646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 14/28] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:25.292261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.292400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.292821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.299824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:25.303372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 15/29] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:25.304510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.312407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:25.314838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:25.392807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 15/30] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:25.397104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.446516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2b8ab-16a3-43d0-b47c-a859dbb945a0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:25.459570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:25.486312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:25.487404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.487832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.488247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.499202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:25.500971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:25.515056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e174-138a-41d0-96a1-f8ba62a50c00 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:34:25.519550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:25.520339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:25.521128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:34:25.521128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:34:25.521128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:34:25.521128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:25.523638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 16/31] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:25.524736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.552254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:25.553003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.553300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.553586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.554368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:25.554665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:25.556122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:25.557438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:25.566145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 16/32] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:25.571071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.633950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e356d-04c0-40eb-b791-b0fb5a47821e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:25.669363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:25.670398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.670924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.671395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.672330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:25.673902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:25.677767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2867-7451-4230-8337-81a7f43db024 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:25.715633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:25.716291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:25.716640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:25.716928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:25.717739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:25.719239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:25.895160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 17/33] 10.222.0.74 () {60 vars in 1131 bytes} [Wed May 25 14:34:25 2022] POST /identity/v3/roles => generated 219 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:25.896635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:25.919645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:25.921784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 17/34] 10.222.0.74 () {60 vars in 1131 bytes} [Wed May 25 14:34:25 2022] POST /identity/v3/roles => generated 225 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:25.922912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.091071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3666fd-31f4-48aa-98b0-8e50c5fd8a91 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.109314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:26.139614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.142159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.142741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.143232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.144489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:26.145025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.147302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.164424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 18/35] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:26.170137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.198386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea5d84-fef1-4275-a92d-2bddd5b89be0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.232169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.233073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.233369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.233656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.238697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.238697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.238697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 18/36] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:25 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:26.244788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.308350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:26.346684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b0c98-8851-450e-b62d-372b1ff590e5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.373799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af596-15c6-4968-aa3b-880925cc6f9a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.387859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.388842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.389134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.389414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.390439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.392308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecb67-1532-4f3b-8dcf-c6156293262c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.400294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:26.413039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.414322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.414654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.414945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.416299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.417935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8253aa-3a06-4f17-8fe4-168936379685 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.420443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 19/37] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:26.421983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b574-2065-4a21-83b5-37b930c78d23 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.421983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b574-2065-4a21-83b5-37b930c78d23 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.423366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b574-2065-4a21-83b5-37b930c78d23 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.424621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 20/38] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:26.427237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b574-2065-4a21-83b5-37b930c78d23 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.519268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 19/39] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/projects => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:34:26.519268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4105a-73a4-44c8-b332-815e3c60c5e6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.519268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4105a-73a4-44c8-b332-815e3c60c5e6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.519268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4105a-73a4-44c8-b332-815e3c60c5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.534671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3fc4105a-73a4-44c8-b332-815e3c60c5e6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:26.536284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3fc4105a-73a4-44c8-b332-815e3c60c5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:26.551488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b574-2065-4a21-83b5-37b930c78d23 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.594659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.595587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.595884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.596174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] PATH_INFO: `/v3/domains/default/users/e695952cd3d8473291efec1acc9a5e16/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.598441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e695952cd3d8473291efec1acc9a5e16, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.600565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.613222 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 20/40] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:26.615805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928419-d40a-4fc7-83c0-324a38241c4b None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.616102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928419-d40a-4fc7-83c0-324a38241c4b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.616434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928419-d40a-4fc7-83c0-324a38241c4b None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.618064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 21/41] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:26.623669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.623964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.624253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.642124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:26.643393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:26.691371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 21/42] 10.222.0.74 () {58 vars in 1361 bytes} [Wed May 25 14:34:26 2022] PUT /identity/v3/domains/default/users/e695952cd3d8473291efec1acc9a5e16/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:34:26.691371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.713919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:26.729811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 22/43] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:26.758505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.773819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:26.837977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65101095-c743-46ac-8e41-d549280d4944 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.884486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.885604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.885604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.885777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.886394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.887874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000e8c-e7c5-4b70-9f9a-4500fee35758 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.890432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17988f9c-47ff-45db-8d67-b297ffef43de None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.924293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.925093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.925382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.925662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.926938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.928465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.930078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 23/44] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:26.935485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.993482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:27.010504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 22/45] 10.222.0.74 () {60 vars in 1131 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/roles => generated 223 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:27.039099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137a15f-5cf7-4846-bf74-7e700e3c62cb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:27.077349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:27.078042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:27.078330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:27.078613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:27.079441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:27.080930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8166f-2296-4eb9-9cd9-f261d3227085 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:27.164589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 24/46] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/projects => generated 312 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:34:28.874981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f7860-f321-4b2f-8c3f-122ee14a3d30 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:28.875526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f7860-f321-4b2f-8c3f-122ee14a3d30 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:28.875526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f7860-f321-4b2f-8c3f-122ee14a3d30 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:28.876949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 23/47] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:28.883035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:28.883429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:28.883612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:28.900088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:28.901275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:28.967744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 25/48] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:28.990745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f7860-f321-4b2f-8c3f-122ee14a3d30 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.090062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f7860-f321-4b2f-8c3f-122ee14a3d30 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.102769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.123394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.124183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.124470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.129396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.135748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:29.143769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:29.144677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:29.145776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 24/49] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:29.151352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.195648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.251652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e06fb-f91d-4e29-a491-32fad5e878fd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.283488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.284146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.284429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.284709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.285450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:29.285761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.287733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c55e-7546-4d9f-a843-0c7841879d41 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.298834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 26/50] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:29.304888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.337033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0bc3f-46bd-449e-8e44-2ef101ff5b73 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.337327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0bc3f-46bd-449e-8e44-2ef101ff5b73 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.337607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0bc3f-46bd-449e-8e44-2ef101ff5b73 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.339036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 27/51] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:29.343815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.344108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.344391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.361237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:29.362439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:29.403642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322834c-e88f-46b1-9371-4ac67da41036 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.429963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 28/52] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:29.437413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.438278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.438565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.438894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.440066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.441511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.443168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 25/53] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:29.448501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.452368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.478918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.493620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.546519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fd52-0d96-41cb-b638-77f3cbc281fa None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.549736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc22f-9b5b-433a-b5e1-34d52bf20a92 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.578657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.579451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.579734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.580007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.581168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.582171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb646b1-2259-48fa-8904-347962556768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.582597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.583012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb646b1-2259-48fa-8904-347962556768 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.583306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb646b1-2259-48fa-8904-347962556768 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.583593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb646b1-2259-48fa-8904-347962556768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.584753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 29/54] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:29.585287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb646b1-2259-48fa-8904-347962556768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.587031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb646b1-2259-48fa-8904-347962556768 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.593131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 26/55] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:29.593242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.593999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb646b1-2259-48fa-8904-347962556768 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.685016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.687917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf354e2-f859-4aa9-8ede-4b2629eaddc5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.690260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb646b1-2259-48fa-8904-347962556768 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.699107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:29.721641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.722258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.722544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.722848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.723675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.domain.id=default, user.id=e695952cd3d8473291efec1acc9a5e16) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:29.723867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.723987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.725110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.725397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.725677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.725956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5c892-c082-4bba-9642-d84fa4d81f6d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.726380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.727843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70abea-60bf-4344-b4ec-b7ef78008cda None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.734580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 30/56] 10.222.0.74 () {56 vars in 1322 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/role_assignments?user.id=e695952cd3d8473291efec1acc9a5e16&scope.domain.id=default&role.id=d38d6cda16454360a4f48025ab4df728 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:29.821546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 27/57] 10.222.0.74 () {60 vars in 1140 bytes} [Wed May 25 14:34:29 2022] POST /identity/v3/projects => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:34:31.991199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75228-62ac-47a9-ba13-01ccaefd8cf8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:31.991744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75228-62ac-47a9-ba13-01ccaefd8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:31.991744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75228-62ac-47a9-ba13-01ccaefd8cf8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:31.993114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 31/58] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:31.998396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:31.998806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:31.999018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:32.015807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:32.017434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:32.081794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 28/59] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:32.107015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75228-62ac-47a9-ba13-01ccaefd8cf8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:32.206364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75228-62ac-47a9-ba13-01ccaefd8cf8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:32.239044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:32.239839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:32.240131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:32.240411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:32.241644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:32.243072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8147e-b7ea-4aef-9459-c959dcdff9de None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:32.244666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 32/60] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:32.250055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:32.340307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:32.349532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a8ce0-2638-48b3-b4a5-18b740cce6a2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:32.382704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:32.383356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:32.383642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:32.383920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:32.384638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:32.386084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bd875-986f-41cb-a623-067aaeeb9ee1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:32.470138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 29/61] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:32 2022] POST /identity/v3/users => generated 275 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:35.675953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb3c5-7cea-4ff7-ae10-59515be29e74 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:35.676974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb3c5-7cea-4ff7-ae10-59515be29e74 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:35.677502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb3c5-7cea-4ff7-ae10-59515be29e74 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:35.679129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 33/62] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:35.684641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9408-9698-4258-ab7b-f69b0faab362 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:35.685471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9408-9698-4258-ab7b-f69b0faab362 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:35.685954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc9408-9698-4258-ab7b-f69b0faab362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:35.711890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-04dc9408-9698-4258-ab7b-f69b0faab362 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:35.713454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-04dc9408-9698-4258-ab7b-f69b0faab362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:35.808920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 30/63] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:35.852186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fb3c5-7cea-4ff7-ae10-59515be29e74 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:35.897487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:35.971016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b690-843b-4bf8-b799-6fd12564adef None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:35.971316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b690-843b-4bf8-b799-6fd12564adef None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:35.971605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377b690-843b-4bf8-b799-6fd12564adef None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:35.972998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 31/64] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:35.980730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:35.981510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:35.981954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:35.996964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fb3c5-7cea-4ff7-ae10-59515be29e74 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.003200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:36.004701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:36.046021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.046848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.047139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.047421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.070153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:36.082587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:36.083476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:36.084787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 34/65] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:36.093224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.109758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 32/66] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:36.147751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.163696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.215531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.226484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c6fbe-42f8-4a30-933c-0c5660e2c76a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.269532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.270607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.270926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.271767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.272803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:36.273116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:36.274774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee0cd-43f2-48e1-a258-a4c129765d78 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:36.289938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 35/67] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:36.289938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b052-6f62-4f52-8899-afca48eb7735 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.289938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b052-6f62-4f52-8899-afca48eb7735 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.289938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b052-6f62-4f52-8899-afca48eb7735 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.289938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 36/68] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:36.294260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225b052-6f62-4f52-8899-afca48eb7735 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.303452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d2046-ea64-44c1-bd7d-a1c888403fe0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.350887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.351794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.352080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.352357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.363238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-1244d92c-ef8f-4b33-ac7e-7fcb26444454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:36.372090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:36.372969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:36.373790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 33/69] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:36.374603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48a1b-baaa-4e4a-80f8-5a077b070039 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.375022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48a1b-baaa-4e4a-80f8-5a077b070039 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.375262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48a1b-baaa-4e4a-80f8-5a077b070039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.391871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fbe48a1b-baaa-4e4a-80f8-5a077b070039 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:36.393060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fbe48a1b-baaa-4e4a-80f8-5a077b070039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:36.414607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225b052-6f62-4f52-8899-afca48eb7735 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.448011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.448882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.449166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.449446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.451479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.458684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 34/70] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:36.463454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3d87-a513-4ce0-a8ae-7435d7bef572 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.463747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3d87-a513-4ce0-a8ae-7435d7bef572 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.464078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d3d87-a513-4ce0-a8ae-7435d7bef572 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.465180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:36.465364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 35/71] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:36.466436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731e3-bd02-4d68-86e2-25d7f266b01e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.466654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0014941-0131-49f0-bacd-20ea6b5c7f04 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:36.466760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731e3-bd02-4d68-86e2-25d7f266b01e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.467014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef731e3-bd02-4d68-86e2-25d7f266b01e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.468387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 36/72] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:36.468483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 37/73] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:36.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef731e3-bd02-4d68-86e2-25d7f266b01e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.476374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.476720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.476954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.494448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:36.495661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:36.502722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.561272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 38/74] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:36.562096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.575105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef731e3-bd02-4d68-86e2-25d7f266b01e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.609613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.610278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.610567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.610886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.611664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:36.611978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:36.613384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831077d4-cc50-4c78-9cc9-37ce670b7614 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:36.621813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 37/75] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:36.623027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.623323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.623608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.640703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:36.641890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:36.668521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec056-ea24-40d4-8855-dc20a2f5df13 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.693989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.703285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.703927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.704762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.704762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.706968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 38/76] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:36.707772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.717507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.774259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:36.775715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:36.804524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f8a7c-5e0a-435f-b181-f4d7a4e09e87 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.837013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.837814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.838103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.838387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.848286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:36.856836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:36.857842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:36.858663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 39/77] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:36.859404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.881800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 39/78] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:36.881800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:36.911217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.919769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:36.959854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3304f-af61-4f58-a863-815e03bab908 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.973360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc854e-f7ae-4d75-beb4-e2d4ff4deb17 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:36.993019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:36.993812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:36.994104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:36.994388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:36.995629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:36.997084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:36.998714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 40/79] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:36.999491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.012324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.013196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.013480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.013755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.014973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.016402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.018015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 40/80] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.018797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.097760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927c9a-1bb5-4ab2-a542-808b7c3e6be5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.113399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.119688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c01d63-2e95-42fb-b19d-5fbdf089f40d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.130198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.131187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.131981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.139753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.139753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.142890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:37.151110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:37.152123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:37.152904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 41/81] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:37.153512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.155302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.156182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.156472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.156759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.157521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:37.157836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.159246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.167905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 41/82] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.171540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.250301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70803eb-9c24-4b9e-91cd-2b5d214a1d9f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.268566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67cac6-da61-4f0e-871e-bcdd255c28e1 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.286180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.287621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.287910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.288198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.289419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.290916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.292611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 42/83] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.293380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.302666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.303342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.303635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.303915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.304630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.306090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.337691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.388926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a819b7-b36f-443e-93f5-43535a5aef75 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.389330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 42/84] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:37 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:37.390259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.423883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.430353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.431107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.431366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.431660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.508049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cb4e2-9497-4c15-8103-e7f6af5a1983 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.509349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.510925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.544460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.545137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.545426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.545704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.546469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:37.546818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.548244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.562368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 43/85] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.562368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.565053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.637035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 43/86] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:37.637939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.672531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.685774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a6e70-cf2a-4a9e-a64b-fef2122db1e0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.733010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.733891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.734181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.734463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.736220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.738038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.740214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 44/87] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.744200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.762487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793ba75-fd52-4511-9b7f-940053cf0ebd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.799120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.799776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.800063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.800349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.801185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:37.801505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.803596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.815013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 44/88] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.815889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.837870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:37.860696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204f4ff-3936-4e14-9a19-b67720a744ed None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.950697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69543d2b-0060-4835-a13b-30c458cd13ec None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:37.961982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.963234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.963707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.964161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:37.971586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:37.973362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:37.975393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 45/89] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:37.976385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:37.997089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:37.998102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:37.998479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:37.998897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.000918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.003510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.007915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 45/90] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:38.008954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.025378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.138784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a774-87fa-4b73-9b65-9fa71597749c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.152737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.184461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925541d-ea22-497b-8148-aa734f292142 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.199522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.200177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.200462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.200801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.249958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.250617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.250926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.251209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.318279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.319776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.355477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.356972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.372774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.483153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.503286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 46/91] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:38.505940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.533855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 46/92] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:38.534970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.606967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba737c6-d6d0-4ac7-ab69-cddedc49dbdd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.642158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.642862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.643155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.643437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.644280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=e695952cd3d8473291efec1acc9a5e16) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:38.644603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.646106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.651727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 47/93] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/role_assignments?user.id=e695952cd3d8473291efec1acc9a5e16&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:38.652555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.723054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.772725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190eb06-d865-4957-8a58-9c87a89f8cc4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.782857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f08-b8b4-4799-b623-16395c17d619 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.821076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.821993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.822285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.822567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.823989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.825490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.826945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.827528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 48/94] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:38.833305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.842114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.843017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.843300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.843577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] PATH_INFO: `/v3/projects/726f4029ad924656bfa3bb000838dcfa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.844934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] RBAC: Authorizing `identity:get_project(project_id=726f4029ad924656bfa3bb000838dcfa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.846398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.848320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 47/95] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/projects/726f4029ad924656bfa3bb000838dcfa => generated 312 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:38.854372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:38.946299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8599b9-6d8b-4710-9a7f-79614f1ae3c2 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:38.956403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:38.983809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:38.984718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:38.985145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:38.985532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:38.986481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:38.986933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:38.988541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997043f8-65db-4645-8a9f-a468d4682168 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:38.995404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 49/96] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=417b7981c02647a98c1d113122c9bdad => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:39.021360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795c4af-81ec-41a1-b31c-5b192dbd59e6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:39.070077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:39.070699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:39.075118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:39.075414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:39.076327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=726f4029ad924656bfa3bb000838dcfa, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:39.076652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:39.078171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58aa8f-1ac4-46d6-85ee-84f853b74983 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:39.084789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 48/97] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:38 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=726f4029ad924656bfa3bb000838dcfa&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:40.241068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de5c8-b0a6-4836-88d2-dc846181b100 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.242102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de5c8-b0a6-4836-88d2-dc846181b100 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.242613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7de5c8-b0a6-4836-88d2-dc846181b100 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.244255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 50/98] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:40.249778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.250585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.251101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.268895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:40.270507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:40.350609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:40.371999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 49/99] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:40.417953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7de5c8-b0a6-4836-88d2-dc846181b100 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:40.536067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7de5c8-b0a6-4836-88d2-dc846181b100 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:40.573688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:40.574496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.574812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.575096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.576350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:40.577859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1318a-a0ec-4d01-8bd7-ca63c0f4a72d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:40.579722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 51/100] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:40.586219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:40.720219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e5ea6-49c4-4dfa-a707-9c3df0eed2b0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:40.769502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:40.770556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.771641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.772124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.773052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:40.774748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2041d2c-31ba-4f1b-8a7d-534fdddf692c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:40.842823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa28a-181c-4036-ad55-059f7461ecbd None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.842823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa28a-181c-4036-ad55-059f7461ecbd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.842823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa28a-181c-4036-ad55-059f7461ecbd None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.844327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 52/101] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:40.849861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.850413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.850923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.869287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:40.870811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:40.889925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 50/102] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:34:40 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:40.915600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:40.959995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 53/103] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:40.983767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534b58-c197-4681-b46c-bb2015f38999 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.984092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534b58-c197-4681-b46c-bb2015f38999 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.984382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06534b58-c197-4681-b46c-bb2015f38999 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:40.985841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 51/104] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:40.987669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:40.998811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:40.999120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:40.999409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.024071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:41.025311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:41.120734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.171148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258527d9-ae0d-4617-b444-d8560992d71e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.222422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.223270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.223551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.223833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.232793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 52/105] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:41.245731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:41.261778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:41.262696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:41.263502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 54/106] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:41.271729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.271868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.322484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.394829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eb857-bc4a-4c08-93e5-a064d4e0b5b9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.457300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.458383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.458998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.461247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434655a-2b4f-437e-93d1-6ae9b902730e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.461716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.482554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:41.491224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:41.492097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:41.492839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:41.495279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 55/107] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:41.501022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.521876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.522357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.522669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.522947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.523703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:41.524015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:41.525386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:41.537609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 53/108] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:41.542829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.579142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.673562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.682771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83709ff-b2cb-447c-bb73-2b1b0e8789e4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.696873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c259b6-9760-4a48-a435-a705624f09ff None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.752926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.753907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.754266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.754617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.756400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:41.758101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:41.764939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 54/109] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:41.770229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.807364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.808324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.808777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.809209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.810232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:41.810755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:41.812343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:41.829930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 56/110] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:41.836438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:41.909446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.913556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-9152-4589-bcfa-cfe37cca96ca None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:41.950816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:41.952029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:41.952512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:41.952982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:41.954390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:41.956125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:41.959111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 55/111] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:41.967795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:42.107066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde952ae-0306-4f16-ab80-6f9b1678271b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.107635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096aede-775c-41b3-a0d0-4ba9a5fa4a3b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.182626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.183536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.183822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.184101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] PATH_INFO: `/v3/users/86a903be097c4cb88c38c1eeba381421` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.185378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.186257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.186535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.186847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/5176609c476e4288aae0b9edac826d58/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.190094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, user_id=5176609c476e4288aae0b9edac826d58, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:42.191570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0a796-5456-4182-9d14-e58c8c01f579 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:42.211132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] RBAC: Authorizing `identity:get_user(user_id=86a903be097c4cb88c38c1eeba381421)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:42.212635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974b127-0e55-465b-b9d5-7a348985ddca None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:42.214354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 57/112] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:41 2022] GET /identity/v3/users/86a903be097c4cb88c38c1eeba381421 => generated 289 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:42.215770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17622939-1354-4282-b846-a690fb9b381b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.216063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17622939-1354-4282-b846-a690fb9b381b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.216344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17622939-1354-4282-b846-a690fb9b381b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.219567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 58/113] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:42.219759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17622939-1354-4282-b846-a690fb9b381b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:42.235804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:42.278957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 56/114] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:41 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/5176609c476e4288aae0b9edac826d58/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:42.280385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2715a-0f81-41d7-b974-a2c8a6f08c5e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.280689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2715a-0f81-41d7-b974-a2c8a6f08c5e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.281031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2715a-0f81-41d7-b974-a2c8a6f08c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.322602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c0a2715a-0f81-41d7-b974-a2c8a6f08c5e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:42.323918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c0a2715a-0f81-41d7-b974-a2c8a6f08c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:42.387033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:42.414333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17622939-1354-4282-b846-a690fb9b381b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.464722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 57/115] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:42.471879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74599801-467f-42cb-b395-686d80479909 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.472183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74599801-467f-42cb-b395-686d80479909 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.472601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74599801-467f-42cb-b395-686d80479909 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.474482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 58/116] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:42.481990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.482403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c647-7948-4f3a-a4b7-688bf1a4a911 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.482682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.482783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c647-7948-4f3a-a4b7-688bf1a4a911 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.482974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.483069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3c647-7948-4f3a-a4b7-688bf1a4a911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.483252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.516858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-60b3c647-7948-4f3a-a4b7-688bf1a4a911 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:42.518149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-60b3c647-7948-4f3a-a4b7-688bf1a4a911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:42.592067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:42.593547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba43a6e-1875-4d72-89a8-ffaa0bf7635c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:42.604961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 59/117] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:42.606523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3875f-821a-4d4b-aac5-3dd4ae717dcf None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.607013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3875f-821a-4d4b-aac5-3dd4ae717dcf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.607443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3875f-821a-4d4b-aac5-3dd4ae717dcf None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.608653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:42.610037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 60/118] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:42.611060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3875f-821a-4d4b-aac5-3dd4ae717dcf None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:42.728106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 59/119] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:42.729627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.729921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.730201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.732103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:42.748210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:42.748543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3875f-821a-4d4b-aac5-3dd4ae717dcf None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.749456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:42.796685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.798412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.798948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.799413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.815045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:42.823776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:42.824625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:42.825407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:42.828189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 61/120] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:42.829225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:42.837556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 60/121] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:42.838670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:42.894035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:42.953768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d68337-a429-4539-aa52-84e67c43f0ff None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.957729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128962a6-9fbd-4e27-96e3-5e723c54fef3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:42.992520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.993245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.993539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.993825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.994085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:42.995212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:42.995584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:42.995922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:42.997198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:42.998775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.001151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 61/122] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:42 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.002002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.011605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:43.023736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:43.024783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:43.025742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:43.028103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 62/123] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:43.028810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.045351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.113985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1209c8-2e57-468c-a483-f21f228a569b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.139169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.140110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9763-0376-454e-b5d3-39bb5506743e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.151390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.152048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.152664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.152664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.153519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:43.153895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.155414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.164696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 62/124] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.165632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.179678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.180755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.181252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.181702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.194497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:43.203558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:43.204426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:43.205204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:43.207800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 63/125] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:43.209525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.272276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b590a-fafc-4794-8a83-d7e230d0a2fa None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.283538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.307413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.308133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.308476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.310907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.310907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=86a903be097c4cb88c38c1eeba381421) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:43.310907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.311781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.317784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 63/126] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/role_assignments?user.id=86a903be097c4cb88c38c1eeba381421&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=33d320a347a54680afe435baa7d666fb => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.318792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.323280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0100afb-1994-4b48-bd5f-1d237f929890 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.357341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.366741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.367780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.379761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.379761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.379761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:43.379761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.379761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.384600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 64/127] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.385750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.435776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272f37-eb53-4da1-83fe-a186fd7c5427 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.501995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.502969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.503212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347053c-cb4f-4293-824f-a038d5e9ee9e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.503323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.503529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.507429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.508259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.510131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 64/128] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.515363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.671262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.671262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.671262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.671262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 65/129] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.671769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.688122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8abe0-c8b3-4323-8dce-cce88014fbe4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.691953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.713877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b405-6c09-4400-9864-c62220522121 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.749954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.751540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.752222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.753052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.756195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.758361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.763191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 65/130] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.764306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.767618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.774660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 66/131] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.775184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.889906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8693-3acc-47a7-bca5-14380e88fd2f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.910098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.928517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.929677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:43.930162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:43.930616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:43.932037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:43.933912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:43.935787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 66/132] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:43.939726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:43.953109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7514-7348-4923-8df0-b89ba053ffb0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:43.981912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:43.989275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:43.990917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.000670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.000670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] PATH_INFO: `/v3/projects/726f4029ad924656bfa3bb000838dcfa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.000670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] RBAC: Authorizing `identity:get_project(project_id=726f4029ad924656bfa3bb000838dcfa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.000670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0ce6-a022-4ea6-96d2-1b3da74f5065 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.000670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 67/133] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:43 2022] GET /identity/v3/projects/726f4029ad924656bfa3bb000838dcfa => generated 312 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:44.002390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec90d9-65c3-442d-971d-59c060c50a64 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.002894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec90d9-65c3-442d-971d-59c060c50a64 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.003384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec90d9-65c3-442d-971d-59c060c50a64 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.007031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 68/134] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:44.007946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec90d9-65c3-442d-971d-59c060c50a64 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:44.053440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7fa8e-3fa7-4700-b490-d270a95f8df9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.099233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.101354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.101906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.102665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/e695952cd3d8473291efec1acc9a5e16/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.105400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, user_id=e695952cd3d8473291efec1acc9a5e16, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.108323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d5de-87bc-4ec5-a1a7-635161296f8e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.140389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec90d9-65c3-442d-971d-59c060c50a64 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.170280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:44.179963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.180810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.181099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.181376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.182673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.190435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.190435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 69/135] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:44 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:44.190435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:44.216248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 67/136] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:43 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/e695952cd3d8473291efec1acc9a5e16/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:44.218230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.218753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.219235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.238803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:44.241014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:44.368030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576adb-c229-45ea-8448-570ade3c1d26 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.383540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 68/137] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:44.384517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:44.445802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:44.449669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.450775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.454664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.454664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] PATH_INFO: `/v3/projects/726f4029ad924656bfa3bb000838dcfa/users/5176609c476e4288aae0b9edac826d58/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.454664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=726f4029ad924656bfa3bb000838dcfa, user_id=5176609c476e4288aae0b9edac826d58, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.455044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.526383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c5f29-a918-4cb5-aa5b-e248b64839aa None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.551011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 70/138] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:44 2022] PUT /identity/v3/projects/726f4029ad924656bfa3bb000838dcfa/users/5176609c476e4288aae0b9edac826d58/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:44.551851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:44.589243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.590331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.590627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.592283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/5176609c476e4288aae0b9edac826d58/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.596075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, user_id=5176609c476e4288aae0b9edac826d58, role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.597701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.651242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:44.683830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 69/139] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:44 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/5176609c476e4288aae0b9edac826d58/roles/417b7981c02647a98c1d113122c9bdad => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:44.718210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f31bd-84e8-438b-b29f-b8e68a452b32 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.760099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.761091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.761379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.761663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.763194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.764722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddf682-b942-4d1e-a429-6f4f558f2f7d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:44.766976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 71/140] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:44.776029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:44.897165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:44.909950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb9ee8-466a-453a-9f47-d2bbeb7074b3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:44.953098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:44.953858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:44.954151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:44.954437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:44.955247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:44.962000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c889a4-2687-44f2-9455-91b740d7a0fc None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:45.065823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 70/141] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:44 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:45.142428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbf2a-cb68-4a8a-96bd-865fff459382 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.142768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbf2a-cb68-4a8a-96bd-865fff459382 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.143033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbf2a-cb68-4a8a-96bd-865fff459382 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.144498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 72/142] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:45.150316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.150617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.150967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.176267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:45.178031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:45.267920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 71/143] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:45.307755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbbf2a-cb68-4a8a-96bd-865fff459382 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:45.440292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbbf2a-cb68-4a8a-96bd-865fff459382 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:45.472616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:45.483928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:45.485701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.485701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.485701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.496879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:45.506306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:45.507234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:45.508014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 73/144] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:45.514327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:45.594694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:45.621312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b654d4-4f03-4a51-85d9-f1c1066e84d8 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:45.659202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:45.659881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.660171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.670439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.670439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:45.670439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:45.670439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:45.672636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 72/145] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:45.679005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:45.786282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeadd8-c92c-4942-aa78-4906aa752617 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:45.830599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:45.831582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.831885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.832167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.833427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:45.835256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:45.837180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 74/146] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:45 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:45.842982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:45.954526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563feab-1d5e-4620-b4e6-3129040f858c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:45.992222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:45.992883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:45.993172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:45.993455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:45.997636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:46.067676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:46.069227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:46.175661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 73/147] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:46.181237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:46.240920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:46.299976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4356ef8-833d-4871-865c-2930cd47b508 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:46.357696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:46.358562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.358870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.359146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.360365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:46.361868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668ae05-c39b-4fde-974c-a6419f192afc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:46.363771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 75/148] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:46 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:46.369599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:46.417050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:46.493670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff33f-fe74-47f6-baea-1e5c762fd0b7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:46.500720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099a54-cf2f-4f80-a39f-d8988fac3873 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.501177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099a54-cf2f-4f80-a39f-d8988fac3873 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.501247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099a54-cf2f-4f80-a39f-d8988fac3873 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.502655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 76/149] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:46.508677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.508973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.509253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.527190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:46.528376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:46.545400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:46.546057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.546343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.546622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.547601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:46.548017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:46.549642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:46.558822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 74/150] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:46 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:46.606070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 77/151] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:46.628637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:46.745199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:46.749053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b8e9-b9c7-4919-aed7-be8fc049afbc None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:46.798234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:46.799075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.799360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.799638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.814090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:46.822550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:46.823463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:46.824198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 75/152] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:46.830964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:46.889186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:46.955377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83173e-cf56-41f3-a33f-ba11d4b0f84a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:46.995451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:46.996353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:46.996859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:46.997281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:46.998243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:46.998738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:47.000297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:47.010970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 78/153] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:47.016428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:47.155820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306000-75b0-4f06-9054-ae5295f13aac None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:47.194436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:47.195344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:47.195628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:47.195908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] PATH_INFO: `/v3/users/86a903be097c4cb88c38c1eeba381421` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:47.199372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] RBAC: Authorizing `identity:get_user(user_id=86a903be097c4cb88c38c1eeba381421)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:47.200856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:47.202905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 76/154] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:47 2022] GET /identity/v3/users/86a903be097c4cb88c38c1eeba381421 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:47.208422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:47.357335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171309f8-bcbd-4b7b-8835-5740bf255561 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:47.375915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:47.402377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:47.403303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:47.403581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:47.403855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:47.405101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:47.406546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd650f-fce8-4358-b840-304be2dcf8db None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:47.413240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 79/155] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:47 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:47.419165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:47.461841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:47.552522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ba117-8b9a-4533-b4f0-9c134c469022 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:47.593048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:47.593890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:47.594171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:47.594445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/86a903be097c4cb88c38c1eeba381421/roles/33d320a347a54680afe435baa7d666fb` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:47.600549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, user_id=86a903be097c4cb88c38c1eeba381421, role_id=33d320a347a54680afe435baa7d666fb)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:47.602021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0055f-f9d0-4da2-acc2-05921cb3ed84 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:47.716802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 77/156] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:47 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/users/86a903be097c4cb88c38c1eeba381421/roles/33d320a347a54680afe435baa7d666fb => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:47.818139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353d64-80a5-4cd5-9fa1-b94f6cee09f5 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:47.818691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353d64-80a5-4cd5-9fa1-b94f6cee09f5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:47.818691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353d64-80a5-4cd5-9fa1-b94f6cee09f5 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:47.820415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 80/157] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:47.827272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:47.827628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:47.827958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:47.860682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:47.861792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:47.956905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 78/158] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:47.996722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96353d64-80a5-4cd5-9fa1-b94f6cee09f5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.120581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96353d64-80a5-4cd5-9fa1-b94f6cee09f5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:48.164584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:48.165385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.165667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.165944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.176624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-e9b76311-ea9b-43dc-b1cc-d968e17755c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:48.184911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:48.185837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:48.186684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 81/159] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:48.194409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.218887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:48.307190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b2def-2b6e-475f-9c6a-e329d026feac None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:48.312449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11a9c7-6118-4cf7-be96-18802a941c44 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.312749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11a9c7-6118-4cf7-be96-18802a941c44 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.313038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11a9c7-6118-4cf7-be96-18802a941c44 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.314704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 82/160] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:48.322228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.322521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.322834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.359224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:48.360656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:48.370737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:48.370737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.370737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.370737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.371145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:48.371145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:48.371145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:48.381742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 79/161] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:48.387487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.443575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:48.512215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 83/162] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:48.531714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:48.553096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6379264-a9da-4bc3-93f5-736e714ba11f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:48.557751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.594499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:48.595815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.596296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.596775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] PATH_INFO: `/v3/users/e695952cd3d8473291efec1acc9a5e16` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.598422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] RBAC: Authorizing `identity:get_user(user_id=e695952cd3d8473291efec1acc9a5e16)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:48.600734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:48.603088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 80/163] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/users/e695952cd3d8473291efec1acc9a5e16 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:48.608739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.717634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81e3bc-1823-4235-8625-c1618f1044b0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:48.720772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880487-0ec2-4cc6-a75f-9a1e99095432 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:48.765040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:48.766542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.767046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.767536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.793670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:48.846797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:48.846797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:48.846797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:48.846797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:48.868305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:48.870100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:48.895581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:48.911982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:48.914028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:48.914830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:48.917819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 84/164] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:48.919211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:48.987328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 81/165] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:34:48.990553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.008493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:49.056487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:49.122175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbed5dc-65d8-4917-ad98-4b0d62106c02 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.173738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.174822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.175718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.176217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.177846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:49.178371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:49.179968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:49.197680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 85/166] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:49.200901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bb2dd-b5ff-4f9d-9319-c809fd1ac473 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.203484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.281851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.282986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.283273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.283550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.285702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:49.287454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1bf4b-dd36-47f2-aa84-01deac9c775e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:49.290719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 82/167] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:48 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:49.292378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf37668-020c-4de3-94a2-eba23552b3b1 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.292870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf37668-020c-4de3-94a2-eba23552b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.293358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf37668-020c-4de3-94a2-eba23552b3b1 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.294858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 83/168] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:49.299120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf37668-020c-4de3-94a2-eba23552b3b1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.432988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:49.439655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf37668-020c-4de3-94a2-eba23552b3b1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.450263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d641-9817-4d2c-a4f8-b7aef521ff3b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.458941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:49.477123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.477840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.478125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.478403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.479341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=e695952cd3d8473291efec1acc9a5e16) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:49.479658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:49.481376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3550861-e29f-40df-a4ed-831082ec8595 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:49.489370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 84/169] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:49 2022] GET /identity/v3/role_assignments?user.id=e695952cd3d8473291efec1acc9a5e16&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:49.491269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.491564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.491846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.520259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:49.521579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:49.544033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.544852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.545139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.545423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.551607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:49.553696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:49.555802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 86/170] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:49 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:49.563611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.650679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 85/171] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:49.688178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.825467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5573e1-ebf2-4727-83a2-5371b8b370d3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.842087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4bc1-b925-49b5-bb5f-00a565ed149f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:49.855214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:49.880831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.881343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:49.881529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.881772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.881981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:49.882084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.882368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:49.882756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:49.893913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:49.902983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:49.903948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:49.904730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 86/172] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:34:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:49.910452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:49.931098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.083694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.085275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0775-932d-4ec5-96d0-7e8a6e720ebd None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.132577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd024513-0c1b-4080-8e7a-6838d5083d9e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.180129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.180893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.181170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.181454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.182215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:50.182530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.191420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828103b-ef36-47f4-b9ae-9309c9d80203 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.201278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 87/173] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:34:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:50.202758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b441c-d17a-4167-b33e-8eabf80bc304 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.203023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b441c-d17a-4167-b33e-8eabf80bc304 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.203315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b441c-d17a-4167-b33e-8eabf80bc304 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.204784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 88/174] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:50.210418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b441c-d17a-4167-b33e-8eabf80bc304 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.219293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 87/175] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:50.220479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.220810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.221096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.242339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:50.243592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:50.256324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.309895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.324661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 88/176] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:50.325766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.370707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b441c-d17a-4167-b33e-8eabf80bc304 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.420937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.422526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.423012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.423831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] PATH_INFO: `/v3/users/5176609c476e4288aae0b9edac826d58` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.425484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] RBAC: Authorizing `identity:get_user(user_id=5176609c476e4288aae0b9edac826d58)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.428522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e04dd-e3b5-4ffc-8d29-a8b8a473c26e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.431325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 89/177] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/users/5176609c476e4288aae0b9edac826d58 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:50.432756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb703-10bf-4491-a99a-503bf939fa59 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.433188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb703-10bf-4491-a99a-503bf939fa59 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.433622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb703-10bf-4491-a99a-503bf939fa59 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.435027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 90/178] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:50.435871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bb703-10bf-4491-a99a-503bf939fa59 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.451520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2aef4-1dbc-4838-82ab-1c54b414c0e2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.488346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.489422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.490298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.490682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] PATH_INFO: `/v3/projects/726f4029ad924656bfa3bb000838dcfa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.491960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] RBAC: Authorizing `identity:get_project(project_id=726f4029ad924656bfa3bb000838dcfa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.493642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.495717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 89/179] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/projects/726f4029ad924656bfa3bb000838dcfa => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:50.496425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.560667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.564857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bb703-10bf-4491-a99a-503bf939fa59 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.570875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.601754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.602569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.602884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.603348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.613981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:50.615604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d0878-f9f3-4cab-b027-1fc917161172 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-7948f6fb-4b19-4407-9f6f-dc496097fab4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:50.622232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:50.623127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:50.623899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 91/180] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:50.625425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.625425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.625425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.643688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:50.644869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:50.654377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.655242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.655371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.655616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.726668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 92/181] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:50.726668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.741842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.743926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059f5bb-ad9d-457d-8d08-cf43d0348a12 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.805168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.819364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:50.831268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4406-071a-46b7-961c-cb01a97d13a0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 90/182] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa49f9-40d2-4729-9daa-0c20ae192c19 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa49f9-40d2-4729-9daa-0c20ae192c19 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa49f9-40d2-4729-9daa-0c20ae192c19 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 91/183] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:50.854026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa49f9-40d2-4729-9daa-0c20ae192c19 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:50.876288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:50.876882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:50.877170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:50.877451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:50.878298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=726f4029ad924656bfa3bb000838dcfa, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:50.878621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:50.880641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:50.886760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 93/184] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=726f4029ad924656bfa3bb000838dcfa&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:50.887404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.063708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa49f9-40d2-4729-9daa-0c20ae192c19 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.107469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.108155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.108447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.108734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.108851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb6fbb-ca8b-4af4-ad8b-38f2a997acf6 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.109499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:51.109817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.111272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.121043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 92/185] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.121882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.154138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.156192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.156993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.157285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.157568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.174835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-bdb468b2-22c0-435f-ac13-9194d6734350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:51.182936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:51.183946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:51.184787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 94/186] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:51.185438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.186347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.186347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.204814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:51.206016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:51.216196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.251982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed8969-1ed2-4faa-b1b2-8f64e3f8e681 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.276485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 95/187] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:51.277868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.290793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.291660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.291946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.292431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.293414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.294912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.296730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 93/188] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.297348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.381347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0bda7-b12e-477a-8f21-aff18ef745f2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.387503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.399100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2290539-0c9e-4ee3-8de5-9411a60e0dad None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.415679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.416491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.416778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.417055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.430567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.432058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de87a2-c5df-49c7-b474-58aa9f2d7bd3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.432460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.433123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.433412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.433704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.433829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 96/189] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.434961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d6c5-1ba4-45ff-abdd-233222ae50f3 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.435251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d6c5-1ba4-45ff-abdd-233222ae50f3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.435532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d6c5-1ba4-45ff-abdd-233222ae50f3 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.435627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:51.435866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.436931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 97/190] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:51.437481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d6c5-1ba4-45ff-abdd-233222ae50f3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.439189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.440093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.448869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 94/191] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.449673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.539203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d6c5-1ba4-45ff-abdd-233222ae50f3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.549589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f9def-fa7e-49a5-bef7-7c5a8c0e8364 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.573751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.574406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.574716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.575003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.575840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=5176609c476e4288aae0b9edac826d58) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:51.576195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.577644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.583403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.583530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 98/192] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/role_assignments?user.id=5176609c476e4288aae0b9edac826d58&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=417b7981c02647a98c1d113122c9bdad => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.584184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.584284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.584538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.584823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.586251 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.587728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c755f-4686-4491-ba4d-cdf9fb684d52 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.589377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 95/193] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.592016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.592309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.592599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.610916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:51.612161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:51.631230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.682949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.690419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 96/194] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:51.693762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.724866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4233-adbf-450f-a4a9-7b7dc0716524 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.761476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.762912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.763192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.763468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.816179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9f89-5a01-488e-9adb-23822578d021 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.848387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.850427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.854505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.855440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:51.855720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:51.855993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:51.857167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:51.858593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:51.860234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 97/195] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:51.862514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.880653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.915219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:51.950424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 99/196] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:51.951212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:51.965565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634357c-f4d6-4149-a7d8-8b78793c2023 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:51.999101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:51.999759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.000044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.000323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.001031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.002472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.046712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dec3-ef75-492b-8245-59a71a7dbf5e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.079043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.079828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.080107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.080384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.090906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:52.098539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:52.100161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:52.101002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 100/197] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:52.101649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.109137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 98/198] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:51 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:52.122139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.135612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.203691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.227179 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876e61d-e9d5-4e5f-a8a6-98113058360d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.237081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6d1e7-928e-4966-b44b-7e6407f3c092 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.264271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.264947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.265235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.265515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.278594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.279609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.279966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.280321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.281630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.283156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.284908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 99/199] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:52.285975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.351432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.352988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.384997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.417389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242763-0441-4ccc-8d9d-039236f61297 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.459788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.460307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.460591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.461119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.461872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:52.462188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.463604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.472900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 100/200] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:52.473744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.475939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 101/201] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:52.476667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.481649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.596759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ad45e-e524-4542-ab7b-a1245bd3f153 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.599173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b19351-be46-48d5-b197-67097dc14fd9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.633572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.642873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.643439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.643681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.644042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.644395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.645050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.645362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=bfc2093064e84ec7ac0f99d2793c4c0c) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:52.646244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.646344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.646818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] PATH_INFO: `/v3/users/86a903be097c4cb88c38c1eeba381421` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.648023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.648420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] RBAC: Authorizing `identity:get_user(user_id=86a903be097c4cb88c38c1eeba381421)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.650234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.652885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 102/202] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/users/86a903be097c4cb88c38c1eeba381421 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:52.653913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.655716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 101/203] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/role_assignments?user.id=bfc2093064e84ec7ac0f99d2793c4c0c&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=417b7981c02647a98c1d113122c9bdad => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:52.658595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.732931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.768942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b133-7607-49e5-96e0-c414fbf68f37 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.776029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48a727-2e4f-48d6-a598-1dceefb24d72 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:52.807348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.807913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.808192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.808472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.819919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:52.821202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:52.821803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:52.822278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:52.823829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.825787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.828201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 103/204] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:52.839940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:52.882777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:52.884351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:52.955464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:52.982880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:53.027771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 102/205] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:53.033249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:53.036841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346a57-4e5c-471d-b7cf-f9b010a07346 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:53.135369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:53.136053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:53.136349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:53.136636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:53.137488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=bfc2093064e84ec7ac0f99d2793c4c0c) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:53.137818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:53.139314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:53.146251 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13ef0-3ca0-4ab7-892c-3c933330dd7c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:53.149723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 104/206] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:52 2022] GET /identity/v3/role_assignments?user.id=bfc2093064e84ec7ac0f99d2793c4c0c&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:53.185950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:53.186840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:53.187129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:53.187405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:53.188601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:53.190054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e5ec6-47d8-4418-967d-63ef6daf4a7e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:53.191793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 103/207] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:53 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:53.198257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:53.360082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be325b82-5dd2-42dd-b78f-a1e016e1d89e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:53.393341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:53.403303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:53.403959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:53.404355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:53.404677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:53.405464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, scope.project.id=50ae2485352046b2a02d4a6d037cf10e, user.id=86a903be097c4cb88c38c1eeba381421) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:53.405782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:53.407324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1470-b378-466e-afa1-779a70abd2dd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:53.414141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 105/208] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:53 2022] GET /identity/v3/role_assignments?user.id=86a903be097c4cb88c38c1eeba381421&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=33d320a347a54680afe435baa7d666fb => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:54.564475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d13-5f45-439d-9ee7-305dcd02d5b6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:54.564982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d13-5f45-439d-9ee7-305dcd02d5b6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:54.565055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d13-5f45-439d-9ee7-305dcd02d5b6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:54.566567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 104/209] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:54.572374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415f79-456a-4397-90ea-ef20e5ecee66 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:54.572782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415f79-456a-4397-90ea-ef20e5ecee66 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:54.572946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415f79-456a-4397-90ea-ef20e5ecee66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:54.591006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2c415f79-456a-4397-90ea-ef20e5ecee66 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:54.592178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2c415f79-456a-4397-90ea-ef20e5ecee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:54.666593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 106/210] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:54.690032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec6d13-5f45-439d-9ee7-305dcd02d5b6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:54.727147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:54.790329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9e3e-38f1-40f1-89f6-06480c2025af None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:54.790677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9e3e-38f1-40f1-89f6-06480c2025af None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:54.790970 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9e3e-38f1-40f1-89f6-06480c2025af None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:54.792682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 107/211] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:54.797445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:54.797744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:54.798060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:54.804680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec6d13-5f45-439d-9ee7-305dcd02d5b6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:54.815462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:54.816916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:54.842424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:54.843166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:54.843449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:54.843726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:54.857472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:54.866022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:54.867159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:54.867956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 105/212] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:54.879987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:54.911911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 108/213] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:54.933296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:54.987937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.009945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.017524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c53a7-2b24-4e38-af20-ffeb3eb199d0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.051951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fed422-d169-455b-99fa-2123cbc59d6f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.066088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.067347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.067894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.068375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.069342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:55.069962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.071676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.081543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 106/214] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:55.088932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.093357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.094571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.094571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.094789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.096025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.097570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.104398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 109/215] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:55.105328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.203475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aedb9-36f8-4e78-b728-7a922bcf1f20 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.237827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6d1a-a58a-4d61-b93e-73b5f2b26d0c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.240085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.241041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.241329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.241610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] PATH_INFO: `/v3/users/5039ef61ab91485d861f1eac0d796019` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.255982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] RBAC: Authorizing `identity:get_user(user_id=5039ef61ab91485d861f1eac0d796019)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.257058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.257441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f162c0-0c87-42fe-ac90-ae92d50ba894 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.259204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 107/216] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/users/5039ef61ab91485d861f1eac0d796019 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:55.260441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34745b-91da-4c0d-8fd2-baacc971c2a2 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.260727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34745b-91da-4c0d-8fd2-baacc971c2a2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.261014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34745b-91da-4c0d-8fd2-baacc971c2a2 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.262413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 108/217] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:55.267098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34745b-91da-4c0d-8fd2-baacc971c2a2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.279285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.280231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.280522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.280811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.281528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.283020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4abf3b-5b53-416e-8bde-ea08395ef8f7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.284464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.373393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34745b-91da-4c0d-8fd2-baacc971c2a2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.402802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 110/218] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:55 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:34:55.404344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.405093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.405590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.417281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.418059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.418492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.418958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.426007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:55.427867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:55.505748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.507318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.526217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.528078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 111/219] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:55.550861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.630817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 109/220] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:55.636651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.672857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc281ce-e340-4142-be09-cf349caa3814 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.676843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.715086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.715915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.716789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.717741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.728944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e53c2b98-9839-4011-8c0e-7d62444cccaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:55.738347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:55.739546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:55.740466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 112/221] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:55.741261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439fb0-a73b-44c5-8d3f-fb69437c7a48 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.741527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439fb0-a73b-44c5-8d3f-fb69437c7a48 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.741810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439fb0-a73b-44c5-8d3f-fb69437c7a48 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.743646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 113/222] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:55.749066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.749362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.749646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.768178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e26a3-c985-4731-8fc3-7bf8ff952b71 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.771208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:55.772459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:55.783399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.809717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:55.810589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:55.810917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:55.812747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:55.814706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:55.815686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:55.821443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 110/223] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:55.821443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.854852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 114/224] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:55.856073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:55.958225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:55.964902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dbd6b-4eaf-4eed-aa00-860d78d803a5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:55.976912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55085e6d-35e7-4cbd-b6e8-e3694ac16599 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.003785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.005656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.006322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.006832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.007984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.008268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=5039ef61ab91485d861f1eac0d796019) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:56.008651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.008936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.009213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.009414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.009958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:56.010265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.011525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.011645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.018002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 115/225] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/role_assignments?user.id=5039ef61ab91485d861f1eac0d796019&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:56.019121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.026151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 111/226] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:56.027465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.151790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:56.179001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4bb5e-734f-4062-8e94-7fb1867bfa65 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.219690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655bd5b-ff3b-4b1a-8849-9c9a66ab4285 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.249968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.250830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.251118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.251720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.261221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.262406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.262870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.263306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.264686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.264814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:56.267101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.269338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 112/227] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:56.274824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:56.275087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:56.276063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:34:56.277019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:56.279394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 116/228] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:56.285533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.307189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:56.395048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86b65a-19db-471a-8769-63abbb035f3e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.401664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33305f95-bb98-417d-b370-b0beadedffd2 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.408898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:56.432096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.433153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.433500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.433851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.435272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.436847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.439263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 113/229] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:56.444708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.445757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.446624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.446963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.447244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.448010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:56.448321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.449720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.460901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 117/230] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:56.467961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.623913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f519e0c-ef26-4efe-8263-7124b0697609 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.627891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:56.662686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fce562-9c81-425f-b391-61ad3a728a51 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.675434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.676449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.676779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.677061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/bfc2093064e84ec7ac0f99d2793c4c0c/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.679751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, user_id=bfc2093064e84ec7ac0f99d2793c4c0c, role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.681220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.697589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.698469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.698810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.699088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.700299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.701747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.708509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 118/231] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:56.709657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.797230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 114/232] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:56 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/bfc2093064e84ec7ac0f99d2793c4c0c/roles/417b7981c02647a98c1d113122c9bdad => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:56.814343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb571-fa9b-43cf-be7e-fdba473b5651 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:56.853042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:56.853930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:56.854220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:56.854502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:56.855830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:56.857326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2caa3d8-5744-486d-913c-59e7b1da923e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:56.862740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 119/233] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:56 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:56.865223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:56.919781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:56.995744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeee3d-51c3-49a5-a4d3-0a2c4b0bb5a2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:57.052113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:57.053131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:57.053419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:57.053695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/bfc2093064e84ec7ac0f99d2793c4c0c/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:57.059604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, user_id=bfc2093064e84ec7ac0f99d2793c4c0c, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:57.061148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ca39-ded9-4066-b9fe-d8930828befd None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:57.174505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 115/234] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:56 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/bfc2093064e84ec7ac0f99d2793c4c0c/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:34:58.150539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570226c0-4869-4909-a478-885eae43f8d7 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.151077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570226c0-4869-4909-a478-885eae43f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.151152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570226c0-4869-4909-a478-885eae43f8d7 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.152765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 120/235] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:58.158237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.158651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.158832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.181145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:58.182341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:58.184632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354ac55-ecc3-4da9-82ba-82fc4a44871a None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.184933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354ac55-ecc3-4da9-82ba-82fc4a44871a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.185220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354ac55-ecc3-4da9-82ba-82fc4a44871a None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.186853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 121/236] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:58.191937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.192252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.192541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.219847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:58.221154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:58.247034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:58.260988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 116/237] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:58.290096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:58.323249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:58.397178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 122/238] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:34:58.403798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b740ede-c0dc-4ea8-b58c-d701f69da833 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:58.437969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:58.438244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:58.439075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.439364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.439639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.441026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:58.443081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:58.444879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 117/239] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:58.450986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:58.559017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:58.561828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e435e-8a3a-4b9c-9ae7-f6ca4032b624 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:58.600436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:58.601129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.601417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.601697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.602416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:58.603891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e008f9-91c7-40ee-8565-1a91e2b3002c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:58.666963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f4fbc-9393-4c95-9b1a-9cf3707a4c91 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:58.702899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:58.703742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.704037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.704319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.722220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 118/240] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:34:58 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:34:58.722910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:58.730132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac34aa-e74e-4231-a987-516a11007992 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.730426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac34aa-e74e-4231-a987-516a11007992 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.730736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac34aa-e74e-4231-a987-516a11007992 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.732125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 119/241] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:58.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:58.735390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:34:58.736202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:34:58.737572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.737866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.738148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.739562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 123/242] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:34:58.747239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:58.773989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:58.776186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:58.780108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:58.920845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 120/243] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:58.947371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758e3a-fbd9-43cc-8e0e-b6267a4310c8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:58.953559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:58.986549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:58.987547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:58.988100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:58.988559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:58.990140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:58.990697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:58.992504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.002850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 124/244] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:59.013295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.065546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.086465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d423d51-c93a-4c1b-a877-2943e2127ad4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.107533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.138820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.140009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.140538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.141035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.149485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb206-1056-4590-ae75-0c60703fcbd5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.158013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:34:59.166974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:34:59.168261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:34:59.169088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:34:59.172882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 121/245] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:34:59.181281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.198170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.199162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.199458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.199742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] PATH_INFO: `/v3/users/a97d6edabaab49ac89878e7ed4108f06` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.219166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] RBAC: Authorizing `identity:get_user(user_id=a97d6edabaab49ac89878e7ed4108f06)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.221276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.226682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 125/246] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/users/a97d6edabaab49ac89878e7ed4108f06 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:59.230224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.333044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245631-6fa4-420e-9839-a3eee29dac99 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.374368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2867-e981-41ea-848e-38432b3c1a33 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.381836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.382983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.383528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.384706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.385730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:34:59.386712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.388430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.400613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 122/247] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:59.407471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.409907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.420027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.420874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.421738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.422282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.422729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.497666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.499484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13d746-5214-4b9b-a645-0e506cdcab71 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.524913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95921cd-b0b0-4c74-94c2-c3d382cda7f9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.563257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.564398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.564785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.565190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] PATH_INFO: `/v3/users/5039ef61ab91485d861f1eac0d796019` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.566594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] RBAC: Authorizing `identity:get_user(user_id=5039ef61ab91485d861f1eac0d796019)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.568234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.570100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 123/248] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/users/5039ef61ab91485d861f1eac0d796019 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:59.575434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.608764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 126/249] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:34:59.610318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b635-02c5-436d-a4b9-ffa1445093c9 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.610810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b635-02c5-436d-a4b9-ffa1445093c9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.611226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b635-02c5-436d-a4b9-ffa1445093c9 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.612997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 127/250] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:59.616384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b635-02c5-436d-a4b9-ffa1445093c9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.660016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.711781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:59.765602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b635-02c5-436d-a4b9-ffa1445093c9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.776614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4e9f9-8365-497e-b8ea-ef0e8e3d76a8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:59.806678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.807774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.808070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.808355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.809677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.811416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8080c-f329-4267-85d1-b7600f46bbf0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.813612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 128/251] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:34:59.815045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.815335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.815860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.836288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:59.837501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:59.910909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 129/252] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:34:59.911600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:59.911803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.912480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:59.912768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:59.913051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:59.915220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:59.916689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:59.918411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 124/253] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:34:59.931576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:59.943918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.030758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b7a0a-1bfc-4008-a377-860f99bf432c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.067659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.068380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.068666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.068939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.069773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=a97d6edabaab49ac89878e7ed4108f06) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:00.070088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:00.071696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:00.077632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 130/254] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:34:59 2022] GET /identity/v3/role_assignments?user.id=a97d6edabaab49ac89878e7ed4108f06&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=33d320a347a54680afe435baa7d666fb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:00.083529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.091643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa39b4-88fc-4d20-82b6-ddaf4034f557 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.154713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.155634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.155923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.156205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/5039ef61ab91485d861f1eac0d796019/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.163448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, user_id=5039ef61ab91485d861f1eac0d796019, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:00.166744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7b5e6-f498-4a46-891d-d52c18086fbd None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:00.199051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.207263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0c8f0-e73e-4a77-a8b0-b440c8f5f473 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.247062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.247888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.248173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.248457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.264330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 125/255] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:34:59 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/5039ef61ab91485d861f1eac0d796019/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:00.265893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b560a1-b125-41b1-b52e-0d436ed953ee None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.271414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b560a1-b125-41b1-b52e-0d436ed953ee None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.271414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b560a1-b125-41b1-b52e-0d436ed953ee None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.271414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 126/256] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:00.274483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.274845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.275204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.279121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:00.286243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:00.301003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:00.302539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:00.303373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 131/257] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:00.309074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:00.311164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.311779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:00.438566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad674d3-8a19-4f3f-bc5e-3c923009a1cd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.478142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.478953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.479249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.479528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.480295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:00.480611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:00.482014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:00.485234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 127/258] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:00.492234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 132/259] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:00.497834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.514166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.528832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.629501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fcc8-7d45-46eb-9b34-c08929a895c9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.667040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.667901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.668193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.668473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.680444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:00.688582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:00.689445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:00.690268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:00.694779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 133/260] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:00.695956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.787045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.817764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141ec18-c638-4eeb-a3bc-3bfaefe73f47 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.849169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358c05-d55a-4402-b593-8adb358ec008 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:00.855948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.856637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.856927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.857210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.857977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:00.858342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:00.860409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:00.877152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 134/261] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:00.878502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:00.956150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:00.963659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:00.964546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:00.964832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:00.965111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:00.966325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:00.967837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:00.969962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 128/262] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:00.975559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.076768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ab54-3e2e-4110-9629-17d60b685822 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.135061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:01.143388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.144235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.144597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.144888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] PATH_INFO: `/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.152834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] RBAC: Authorizing `identity:get_user(user_id=bfc2093064e84ec7ac0f99d2793c4c0c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.154419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:01.156247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 135/263] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/users/bfc2093064e84ec7ac0f99d2793c4c0c => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:01.162556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.181193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12550463-df5b-4516-93b3-75a4a56861ba None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.221203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.222172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.222604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.222913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.319134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66406ca-95ed-482d-9c5e-d6bf2f13e121 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.360347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.361002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.361295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.361665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.381674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:01.388353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.389855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:01.433104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:01.436425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.438048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:01.532420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 129/264] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:01.543637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.547447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 136/265] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:01.548773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.652011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:01.655918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a582-e98a-40ff-af1b-d3c92996933d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.701950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56079a-2aee-4556-a900-0bd479229034 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.703897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.705041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.705502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.705956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.707374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.714660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:01.715000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:01.717324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 137/266] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:01 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:01.723160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.756369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.757340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.757697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.758056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.759368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.760898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:01.771339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 130/267] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:01 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:01.776824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:01.926132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018e3d9-16b6-453c-9ac6-a45318f7f298 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.933747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9cc1-4c67-4064-a760-4157e912e2fc None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:01.985224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:01.985898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:01.986187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:01.986470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:01.987343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=bfc2093064e84ec7ac0f99d2793c4c0c) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:01.987664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:01.989650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4336b11-3489-4119-bd8e-31720e2ed37e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:02.004444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 131/268] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:01 2022] GET /identity/v3/role_assignments?user.id=bfc2093064e84ec7ac0f99d2793c4c0c&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=417b7981c02647a98c1d113122c9bdad => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:02.004623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:02.005705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.005993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.006270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.007131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=bfc2093064e84ec7ac0f99d2793c4c0c) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:02.007450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:02.008924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e7d50-4f93-431e-bbf4-fe34b2efc454 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:02.027861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 138/269] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:01 2022] GET /identity/v3/role_assignments?user.id=bfc2093064e84ec7ac0f99d2793c4c0c&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:02.438128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4aca-75bf-4ccb-9f32-5adb62034b8a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.438624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4aca-75bf-4ccb-9f32-5adb62034b8a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.438720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4aca-75bf-4ccb-9f32-5adb62034b8a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.440377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 132/270] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:02.445916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.446358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.446532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.453480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f4512-d1f4-417d-9cee-74bb8e04a4ab None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.453763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f4512-d1f4-417d-9cee-74bb8e04a4ab None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.454042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f4512-d1f4-417d-9cee-74bb8e04a4ab None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.455728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 133/271] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:02.461181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.461727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.462223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.469465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:02.470758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:02.481764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:02.483306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:02.513798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:02.519931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:02.570691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 139/272] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:02.574685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 134/273] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:02.597989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:02.600382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:02.728940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defbc8b-fd14-43e3-926d-afdc2054d5e2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:02.733577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4683e3-6dc4-4754-b385-fb2419c30e3b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:02.775308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:02.775724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:02.776196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.776483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.776756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.777882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.778681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.780240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.781803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:02.783591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:02.785688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 135/274] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:02.790915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:02.791947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:02.797202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:02.798097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:02.798937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:02.799641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 140/275] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:02.805461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:02.833672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:02.846987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:02.909937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627602ca-68cd-4e3b-b96b-51c50395d868 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:02.946439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f08b-a318-4450-be14-f211b14b4aea None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:02.950807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:02.950807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.950807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.950807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.951133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:02.952644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b19a24-f7df-47b9-92b6-c86af0885a91 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:02.990177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:02.990886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:02.991177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:02.991461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:02.992220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:02.992533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:02.994046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:03.002983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 141/276] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:03.008708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.071020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 136/277] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:02 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:03.108829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.126976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a763f8-bfe4-45a7-8bb4-00d3fda0ccef None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.152893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec405-2761-4c8b-8e7b-b81406b94e1e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.153211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec405-2761-4c8b-8e7b-b81406b94e1e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.153465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfec405-2761-4c8b-8e7b-b81406b94e1e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.155571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 137/278] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:03.162252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.162585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.162984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.172251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.173500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.173975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.174739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] PATH_INFO: `/v3/users/1f4babaa4456407da8135677cf455337` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.184806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:03.186188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:03.192940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] RBAC: Authorizing `identity:get_user(user_id=1f4babaa4456407da8135677cf455337)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:03.194954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:03.198378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 142/279] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/users/1f4babaa4456407da8135677cf455337 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:03.204875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.282040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 138/280] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:03.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.320474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.325535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83b749-124e-49f6-b807-fd60e6af9f6f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.365152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.365817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.366104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.366387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.367368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, user.id=1f4babaa4456407da8135677cf455337, scope.system=all) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:03.367742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:03.369217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683e877-d388-4882-afad-c5c71c880fd2 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:03.382735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 143/281] 10.222.0.74 () {56 vars in 1308 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/role_assignments?user.id=1f4babaa4456407da8135677cf455337&scope.system=all&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:03.385367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae767-8696-4f44-b21a-bedcff43c55f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.385661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae767-8696-4f44-b21a-bedcff43c55f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.385948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae767-8696-4f44-b21a-bedcff43c55f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.387762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 144/282] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:03.393216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.393504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.393786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.408844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.410204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c6adc-9533-4b98-b94c-d6d5fa1cd3ea None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.413372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:03.414541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:03.454814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.455640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.455923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.456200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.468439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:03.476443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:03.477391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:03.478163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 139/283] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:03.483858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.512602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 145/284] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:03.537995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.589121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.616071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788d1-91c4-4db6-94ab-1ab751b20ecf None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.648921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bd61e-fe8d-4087-adb2-964ad74e0570 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.665554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.666462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.667000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.669373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.670315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:03.671041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:03.673838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:03.691628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 140/285] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:03.700441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.716153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.717232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.717661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.718080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.723116 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.736212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:03.744790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:03.745842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:03.746658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:03.747776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 146/286] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:03.754269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.835211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cb29f-4b88-4921-981c-24412a1e41b9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:03.884791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:03.886038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:03.886544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:03.887309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] PATH_INFO: `/v3/users/a97d6edabaab49ac89878e7ed4108f06` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:03.890435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a97d6edabaab49ac89878e7ed4108f06)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:03.892161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:03.894349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 141/287] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/users/a97d6edabaab49ac89878e7ed4108f06 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:03.899461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:03.918495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:03.953603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68848-9947-402d-b1e4-1f268d948c33 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.017095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.017773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.018063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.018345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.019140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:04.019455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.021412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.039871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 147/288] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:04.045418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:04.066779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a61e0-f8f1-4f6e-bba1-70b9721e6fb9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.108252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.109665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.110358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.110854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.112283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.114071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.117060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 142/289] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:03 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:04.124447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:04.178732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:04.251201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:04.275278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12401547-e203-4e24-a09f-54e4a2d57fc3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.277681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32def-1e4b-4c55-8792-9ddb9a278d57 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.324120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db341-ee1a-4902-af24-229181744fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.325014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db341-ee1a-4902-af24-229181744fac None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.325300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db341-ee1a-4902-af24-229181744fac None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.325580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db341-ee1a-4902-af24-229181744fac None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/a97d6edabaab49ac89878e7ed4108f06/roles/33d320a347a54680afe435baa7d666fb` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.330034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538db341-ee1a-4902-af24-229181744fac None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, user_id=a97d6edabaab49ac89878e7ed4108f06, role_id=33d320a347a54680afe435baa7d666fb)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.336269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538db341-ee1a-4902-af24-229181744fac None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.345163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.346101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.346390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.346688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] PATH_INFO: `/v3/users/5039ef61ab91485d861f1eac0d796019` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.347943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] RBAC: Authorizing `identity:get_user(user_id=5039ef61ab91485d861f1eac0d796019)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.350160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.352249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 148/290] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:04 2022] GET /identity/v3/users/5039ef61ab91485d861f1eac0d796019 => generated 297 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:04.358680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:04.429244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 143/291] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:04 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/users/a97d6edabaab49ac89878e7ed4108f06/roles/33d320a347a54680afe435baa7d666fb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:04.464845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09261f2e-6d43-42a2-aa01-69eb99c4a8e1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.492937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:04.502770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.507724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.507724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.507724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.577055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.578571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.690078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 149/292] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:04.695932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db341-ee1a-4902-af24-229181744fac None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:04.825619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db341-ee1a-4902-af24-229181744fac None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:04.862401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:04.863293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:04.863577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:04.864412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:04.865613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:04.867078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a82b6c-16b4-4e59-8594-a666ddb34b15 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:04.868765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 144/293] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:04 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:04.874518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:04.903965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:05.010376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dac97-2f72-43f7-8dc3-84541e5d8e22 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:05.051952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:05.052616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:05.052904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:05.053180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:05.054013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=5039ef61ab91485d861f1eac0d796019) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:05.054377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:05.055903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dc71f-ed02-4262-82bb-b25a428e347b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:05.065821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 150/294] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:04 2022] GET /identity/v3/role_assignments?user.id=5039ef61ab91485d861f1eac0d796019&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:05.763269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85878-36ba-4928-bfc8-1f37ea34cdab None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:05.763818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85878-36ba-4928-bfc8-1f37ea34cdab None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:05.763818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85878-36ba-4928-bfc8-1f37ea34cdab None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:05.765434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 145/295] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:05.771514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:05.771936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:05.772123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:05.791449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:05.792646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:05.842304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb867-645f-46e8-8c9d-4f90cf2d542e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:05.842616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb867-645f-46e8-8c9d-4f90cf2d542e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:05.842946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb867-645f-46e8-8c9d-4f90cf2d542e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:05.844671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 146/296] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:05.850366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:05.850674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:05.850964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:05.856119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:05.867707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 151/297] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:05.875789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:05.875789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:05.891404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:05.917818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:05.940120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 147/298] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:05.968633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.022855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23650e91-2ff7-4649-97e4-75d89e584344 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.058027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.058862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.059147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.059473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.070785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:06.078726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:06.079584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:06.080348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:06.084247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f2ada-d130-4ece-a11a-bfcac280fb02 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.085309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 152/299] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:06.086514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.117607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:06.123519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.124517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.125047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.138901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.138901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.138901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.138901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 148/300] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:06.138901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.170443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:06.189762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b91b5-7485-481e-bc1e-8049abbe307b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.228067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.228754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.229042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.229323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.230071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:06.230386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.231838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe9602-8cea-4d3c-b631-a47ed0c7ce55 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.239882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 153/301] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:06.244607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c695-9600-42ea-837d-dd38f6149e49 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.244607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c695-9600-42ea-837d-dd38f6149e49 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.244607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c695-9600-42ea-837d-dd38f6149e49 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.244607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 154/302] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:06.245163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c695-9600-42ea-837d-dd38f6149e49 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.260662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6129-73e3-406a-83b9-376310bcf031 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.297470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.298753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.299254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.299713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] PATH_INFO: `/v3/groups` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.300778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.303128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae22018-1633-4d38-a46b-d8ff231053e7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.350431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c695-9600-42ea-837d-dd38f6149e49 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.361863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 149/303] 10.222.0.74 () {60 vars in 1134 bytes} [Wed May 25 14:35:06 2022] POST /identity/v3/groups => generated 229 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:35:06.372543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.373539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.374007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.396121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.397061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.397348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.397622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] PATH_INFO: `/v3/users/f7251c1c87734037844efae3ee660e00` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.402177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:06.403778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:06.412250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:06.415266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] RBAC: Authorizing `identity:get_user(user_id=f7251c1c87734037844efae3ee660e00)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.416857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.422687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 155/304] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/users/f7251c1c87734037844efae3ee660e00 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:06.425560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.490322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:06.504174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 150/305] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:06.532954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.560049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47235c-5c1f-409b-bf77-879c65b4736f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.609876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.610534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.610873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.611362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.611953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, user.id=f7251c1c87734037844efae3ee660e00, scope.system=all) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:06.612281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.613703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.620716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 156/306] 10.222.0.74 () {56 vars in 1308 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/role_assignments?user.id=f7251c1c87734037844efae3ee660e00&scope.system=all&role.id=33d320a347a54680afe435baa7d666fb => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:06.662334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516fd83-469b-4c36-a47e-c316585bb592 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.702101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.702950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.703230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.703503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.715959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:06.724524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:06.725495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:06.726286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 151/307] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:06.732039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.800231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:06.841854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da61318-4ddf-48c1-bfa6-979e0566633e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:06.877103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:06.877770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:06.878053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:06.878330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:06.879107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:06.879415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:06.880849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:06.889379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 157/308] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:06.895700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:06.952631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:07.012560 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8994d-1cfa-459d-bb97-6167969a74d4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.049196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.050459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.050980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.051513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] PATH_INFO: `/v3/users/1f4babaa4456407da8135677cf455337` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.052915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] RBAC: Authorizing `identity:get_user(user_id=1f4babaa4456407da8135677cf455337)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:07.054569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826dfcc-92ae-4361-8542-bda5991a39a3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:07.056728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 152/309] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:06 2022] GET /identity/v3/users/1f4babaa4456407da8135677cf455337 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:07.061916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:07.093546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01cc8b-37e2-479d-bb22-c6b616b07b66 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.093785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01cc8b-37e2-479d-bb22-c6b616b07b66 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.094025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01cc8b-37e2-479d-bb22-c6b616b07b66 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.095562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 153/310] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:07.103900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.104189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.104470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.122435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:07.123774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:07.179537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b22b76-dffb-4f3b-9178-e4b2fdeac5aa None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.206490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 154/311] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:07.206812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:07.224371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.225242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.225583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.225870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] PATH_INFO: `/v3/system/users/1f4babaa4456407da8135677cf455337/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.227461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f4babaa4456407da8135677cf455337, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:07.228921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:07.230686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:07.280720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:07.336874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 158/312] 10.222.0.74 () {58 vars in 1334 bytes} [Wed May 25 14:35:07 2022] PUT /identity/v3/system/users/1f4babaa4456407da8135677cf455337/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:07.347685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69ad41-365e-4cd0-9784-462824d49f80 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.388806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.389616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.389901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.390228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.403270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:07.411946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:07.412897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:07.413687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 155/313] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:07.419156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:07.534931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4af41-19e7-452b-a280-019eabb04126 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.569459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.570131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.570417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.570727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.571472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:07.571784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:07.573172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:07.581633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 159/314] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:07.588043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:07.693744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a8a4-0a1a-4300-b64e-0b7be4006fcb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.703301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:07.729488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] PATH_INFO: `/v3/users/a97d6edabaab49ac89878e7ed4108f06` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] RBAC: Authorizing `identity:get_user(user_id=a97d6edabaab49ac89878e7ed4108f06)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:07.736183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 156/315] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:07 2022] GET /identity/v3/users/a97d6edabaab49ac89878e7ed4108f06 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:07.741549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:07.802800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:07.853372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c6c32-0c6c-4a52-9feb-dced3aa9d712 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:07.888922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:07.889573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:07.889858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:07.890190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.007725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:08.009442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:08.061769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:08.121027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 160/316] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:08.123722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:08.232973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93f29c-b5f9-4c17-a40c-55c359603375 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:08.273258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:08.274051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.274339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.274624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.275866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:08.277327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d283433-a9da-4227-a12d-b3964deab187 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:08.279366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 157/317] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:08 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:08.284988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:08.389894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484e07c-3087-46a3-96f6-0ccb1506b3c7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:08.429142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:08.429810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.430097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.430389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.431271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727, user.id=a97d6edabaab49ac89878e7ed4108f06) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:08.431594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:08.433062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7451082-f03c-4399-83f0-669e4267ea29 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:08.439478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 161/318] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:08 2022] GET /identity/v3/role_assignments?user.id=a97d6edabaab49ac89878e7ed4108f06&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=33d320a347a54680afe435baa7d666fb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:08.704579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113477be-b748-4ea8-afee-972633b44407 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.704745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113477be-b748-4ea8-afee-972633b44407 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.705001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113477be-b748-4ea8-afee-972633b44407 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.706486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 158/319] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:08.712759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f31fc2-7cf2-44f1-8499-01582d4297dc None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.713115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f31fc2-7cf2-44f1-8499-01582d4297dc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.713466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f31fc2-7cf2-44f1-8499-01582d4297dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.731635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e9f31fc2-7cf2-44f1-8499-01582d4297dc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:08.732875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e9f31fc2-7cf2-44f1-8499-01582d4297dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:08.746282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:08.806732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 162/320] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:08.828716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113477be-b748-4ea8-afee-972633b44407 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:08.876671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:08.933316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113477be-b748-4ea8-afee-972633b44407 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:08.939352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c92e3-4b01-4968-835c-57130488a520 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.939650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c92e3-4b01-4968-835c-57130488a520 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.939988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c92e3-4b01-4968-835c-57130488a520 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.941367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 163/321] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:08.947043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.947332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.947826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.965590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:08.967269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:08.969460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:08.970250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:08.970525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:08.970833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:08.974252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:08.975822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:08.977500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 159/322] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:08.982972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.042552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 164/323] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:09.064966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.088466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4135b57-c001-4334-9839-bb4bf7ddac26 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:09.103338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:09.128772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:09.130212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.130504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.130872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] PATH_INFO: `/v3/groups` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.131806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:09.133291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:09.147510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:09.170071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789362b-3e4f-43bd-a6dd-a595fa216d31 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:09.204069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:09.204876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.205161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.205440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.216320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:09.224541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:09.225432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:09.226237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 165/324] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:09.231807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.236432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 160/325] 10.222.0.74 () {60 vars in 1134 bytes} [Wed May 25 14:35:08 2022] POST /identity/v3/groups => generated 226 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:09.355600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337797f7-cb89-4f11-a9de-cbd69c752b6d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:09.393016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:09.393697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.393984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.394261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.395613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:09.395928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:09.397344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:09.400527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:09.414075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 166/326] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:09.415617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.526237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c4c7-a8c4-4871-8c60-b0657a4b986a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:09.563915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:09.564800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.565084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.565356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] PATH_INFO: `/v3/users/f7251c1c87734037844efae3ee660e00` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.566582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] RBAC: Authorizing `identity:get_user(user_id=f7251c1c87734037844efae3ee660e00)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:09.568317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853859-8693-48bd-a56e-5d054043dcfc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:09.569963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 161/327] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:09 2022] GET /identity/v3/users/f7251c1c87734037844efae3ee660e00 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:09.575559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.683381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab68a4-8730-4ecd-a90d-d95e638b4643 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:09.706178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf810-86e3-4097-9c23-509be5b82585 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.706475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf810-86e3-4097-9c23-509be5b82585 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.706841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf810-86e3-4097-9c23-509be5b82585 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.708800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 162/328] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:09.715556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.715851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.717063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.727083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:09.727956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:09.728238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:09.728517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] PATH_INFO: `/v3/system/users/f7251c1c87734037844efae3ee660e00/roles/33d320a347a54680afe435baa7d666fb` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:09.730049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f7251c1c87734037844efae3ee660e00, role_id=33d320a347a54680afe435baa7d666fb)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:09.731492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:09.738018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:09.739433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:09.783707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:09.797391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 167/329] 10.222.0.74 () {58 vars in 1334 bytes} [Wed May 25 14:35:09 2022] PUT /identity/v3/system/users/f7251c1c87734037844efae3ee660e00/roles/33d320a347a54680afe435baa7d666fb => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:09.815691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 163/330] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:09.842222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:09.918795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:10.005003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bb28-b6ea-4dd8-80e4-4a2aef36f4f8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:10.052508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:10.053250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:10.053540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:10.053820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:10.066832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:10.074800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:10.075711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:10.076910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 168/331] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:10.085741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:10.198584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b27b8-596d-49f1-a0b8-6840d0a8b81d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:10.240585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:10.241250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:10.241530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:10.241804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:10.242551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:10.242886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:10.244256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:10.253055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 164/332] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:10.259724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:10.409632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6634c0-7526-45db-be68-31f54a8385ff None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:10.437320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:10.450141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] PATH_INFO: `/v3/users/1f4babaa4456407da8135677cf455337` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] RBAC: Authorizing `identity:get_user(user_id=1f4babaa4456407da8135677cf455337)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636988f-1d7a-4008-b828-db9e871c875a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:10.460481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 169/333] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:10 2022] GET /identity/v3/users/1f4babaa4456407da8135677cf455337 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:10.465126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:10.522157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:10.609717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4deb-12ad-4934-908d-4c3f471c4520 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:10.648557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:10.649213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:10.649494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:10.649770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:10.650598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] RBAC: Adding query filter params (role.id=3f2e67fc62664875876c5e7f51498b6d, user.id=1f4babaa4456407da8135677cf455337, scope.system=all) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:10.650970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:10.652411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11c72-92e5-4c81-a3fa-6fb436543cac None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:10.665276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 165/334] 10.222.0.74 () {56 vars in 1308 bytes} [Wed May 25 14:35:10 2022] GET /identity/v3/role_assignments?user.id=1f4babaa4456407da8135677cf455337&scope.system=all&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:12.310828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699433-de86-4800-927c-0ff9f1529983 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.310828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699433-de86-4800-927c-0ff9f1529983 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.310828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699433-de86-4800-927c-0ff9f1529983 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.310828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 170/335] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:12.324766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d4582-bb94-4172-b8a8-792953956eca None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.331127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d4582-bb94-4172-b8a8-792953956eca None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.331617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d4582-bb94-4172-b8a8-792953956eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.355901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-094d4582-bb94-4172-b8a8-792953956eca None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:12.357417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-094d4582-bb94-4172-b8a8-792953956eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:12.440316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 166/336] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:12.484806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699433-de86-4800-927c-0ff9f1529983 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:12.611745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699433-de86-4800-927c-0ff9f1529983 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:12.660438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:12.661813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.662097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.662375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.679673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-329216ea-a82b-406d-855a-d18689d4a46b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:12.688151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:12.689055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:12.689839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:12.690980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 171/337] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:12.695731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d4582-bb94-4172-b8a8-792953956eca None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:12.697259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012984-15a8-485c-8816-561ca047977e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.697555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012984-15a8-485c-8816-561ca047977e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.697839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012984-15a8-485c-8816-561ca047977e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.699301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 172/338] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:12.704960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.705255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.705539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.734274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:12.734970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:12.735510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:12.821722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:12.858462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d4582-bb94-4172-b8a8-792953956eca None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:12.892385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 173/339] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:12.908123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:12.908801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:12.909079 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:12.909360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:12.910111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:12.913056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:12.913056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:12.927175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 167/340] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:12.933055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:12.938404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:13.057041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.074763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858435d-37f5-467c-9bdd-1abd9ac9e38d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:13.132232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb65c11-c379-423c-8f8d-de51b8a18d36 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:13.137735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:13.138855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.139343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.139801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.168504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5e16048f-fdc7-45d7-a720-c929de9a8670 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:13.177119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:13.178078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:13.178879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:13.189667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 168/341] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:13.192244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f616b4-7f54-47cd-b4d0-1f05b42248d5 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.193391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f616b4-7f54-47cd-b4d0-1f05b42248d5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.193975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f616b4-7f54-47cd-b4d0-1f05b42248d5 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.196837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 169/342] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:13.198608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f616b4-7f54-47cd-b4d0-1f05b42248d5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:13.215292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:13.216146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.222674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.222674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.334693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.360618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:13.362353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2810932-7e01-43f1-b0a9-10e1a7dca11b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:13.425586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f616b4-7f54-47cd-b4d0-1f05b42248d5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:13.467967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:13.469007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.469464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.469911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.470858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:13.471396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:13.472955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87076-1367-4966-9ac7-cffcc5f80be3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:13.484645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 170/343] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:13.485744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 174/344] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:13.486364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f298b4-2fbc-485b-82ae-1b5bd865a260 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.486841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f298b4-2fbc-485b-82ae-1b5bd865a260 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.487098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eb79-0196-4b7c-aee9-16cf159951cb None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.487385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eb79-0196-4b7c-aee9-16cf159951cb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.487661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eb79-0196-4b7c-aee9-16cf159951cb None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.487819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f298b4-2fbc-485b-82ae-1b5bd865a260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.489728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 175/345] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:13.490433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9eb79-0196-4b7c-aee9-16cf159951cb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:13.496763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.508113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-82f298b4-2fbc-485b-82ae-1b5bd865a260 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:13.510131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-82f298b4-2fbc-485b-82ae-1b5bd865a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:13.591572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 171/346] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:13.593121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.593844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.594316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.598792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.619165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:13.620695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:13.624738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9eb79-0196-4b7c-aee9-16cf159951cb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:13.669469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:13.670123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.670713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.670713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.706068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 172/347] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:13.707037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:13.761831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:13.763732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:13.800586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.851964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3a64b-1362-4f68-8955-e4095d9ff498 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:13.876707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:13.896491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:13.897421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:13.897703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:13.897976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:13.899349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:13.900627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:13.902725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 173/348] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:13 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:13.903290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 176/349] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:13.903502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:13.904617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.026958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6221af-f930-4774-bad5-948dabecd635 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.041772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a147d-f2f7-4088-8d8e-36e7ff49ff1e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.069222 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.078143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.079010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.079283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.079551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.088363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.092054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.092054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.092054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.099070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:14.106939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:14.107870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:14.108682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 174/350] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:14.109327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.110140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:14.118095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:14.119070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:35:14.120299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:14.121122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 177/351] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:14.122140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.178234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.241170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9e94-6580-4b80-9274-e060afe938a9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.257528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1786b-7ace-415a-bc12-fa0fb0d73563 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.284593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.285504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.286307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.286585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.289041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.290532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a447e-a225-4b6c-a63d-c4f4d8afb57c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.292556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 175/352] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.293701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a6cc-801b-4602-95f0-60141718fb63 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.294062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a6cc-801b-4602-95f0-60141718fb63 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.294346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7a6cc-801b-4602-95f0-60141718fb63 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.295759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 176/353] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:14.296522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7a6cc-801b-4602-95f0-60141718fb63 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.307340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.308647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.309130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.309410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.321479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.323163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.325298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 178/354] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.326111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.401210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.452245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7a6cc-801b-4602-95f0-60141718fb63 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.459690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c493-74a4-4f1b-9c41-e986a5a4e604 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.464295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.499847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.500995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.501283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.501613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.502321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:14.502624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.502755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.505814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.505814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.505814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.506063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.506139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:14.506139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.506452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ac700-efb3-4729-a73b-fdc521640e67 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.514037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 177/355] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.514728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.515157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 179/356] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.516612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.516908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.517228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.534741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:14.536214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:14.607368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 180/357] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:14.608509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.631928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c805e5-5641-484f-8fcb-613aae5ba53b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.671496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.672357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.672644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.672916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.674185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.676386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a7219-2389-430d-8b20-e2ab9a60e8f4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.679029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.679317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 178/358] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.682845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ddbfd-ffec-47e2-9ea5-e03e5f0d0d76 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.682845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ddbfd-ffec-47e2-9ea5-e03e5f0d0d76 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.682845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ddbfd-ffec-47e2-9ea5-e03e5f0d0d76 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.683175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 179/359] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:14.687565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ddbfd-ffec-47e2-9ea5-e03e5f0d0d76 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.726165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:14.732127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708588-2385-4556-813c-3c2129b926e4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.769339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.770535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.770847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.771122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.771949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=50ae2485352046b2a02d4a6d037cf10e) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:14.772258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.773820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.781629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 181/360] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=417b7981c02647a98c1d113122c9bdad => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.782459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.817267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ddbfd-ffec-47e2-9ea5-e03e5f0d0d76 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.874811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.876865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.877008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.877275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] PATH_INFO: `/v3/users/f7251c1c87734037844efae3ee660e00` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.880245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=f7251c1c87734037844efae3ee660e00)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:14.883066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:14.885449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 180/361] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/users/f7251c1c87734037844efae3ee660e00 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:14.886453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:14.924850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be357af-40a4-4b7a-bd41-4327d3480380 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:14.964237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:14.964924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:14.965209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:14.965486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:14.971526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.002008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.023975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08eed7-1f4d-4ee7-8a8b-8aebebddc4a1 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.054881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.058598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414dc20-ddfb-412f-8654-ab53fa4f413f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.064770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.065591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.065875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.066329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.080644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:15.089665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:15.090685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:15.091690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 181/362] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:15.092538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.190932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 182/363] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:15.192908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.193202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.193485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.213914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:15.215137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:15.225084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765221-bb1d-4754-8989-3780ec1e8386 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.246220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.269136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.270726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.270944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.271223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.272071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=50ae2485352046b2a02d4a6d037cf10e) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:15.272387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.273822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.280689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 182/364] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:15.281539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.294253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.308379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 183/365] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:15.309726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.396500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aadba-d883-46d4-b51c-5e0caa3c16ab None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.432160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fa36a-4a9b-48ba-a7d9-a98653d525c5 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.434015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.434715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.435002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.435280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.436124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] RBAC: Adding query filter params (role.id=33d320a347a54680afe435baa7d666fb, user.id=f7251c1c87734037844efae3ee660e00, scope.system=all) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:15.436436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.438125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.452228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 183/366] 10.222.0.74 () {56 vars in 1308 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/role_assignments?user.id=f7251c1c87734037844efae3ee660e00&scope.system=all&role.id=33d320a347a54680afe435baa7d666fb => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:15.458946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.475381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.476221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.476510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.476788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.477552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:15.477864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.479996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.491994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 184/367] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:15.492809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.555061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.563331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.630721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7048a1-4cdd-4bad-aafe-74129af6d9a8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.632564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970803-4f19-47b9-b5c8-473d5794d3e0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.671975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.672781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.673060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.673337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.680108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.681277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.681568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.682023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.683469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.685297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.687127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 184/368] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:15.689320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.690287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:15.698300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:15.699292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:15.700066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 185/369] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:15.700722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.823414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac03112-6339-4581-901d-c1fa6d7eeee3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.833152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.836464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1288e-acc8-424e-8e19-a4c12062f64e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:15.853592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:15.863379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.864656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.864937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.865206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.882505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:15.883346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:15.883628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:15.883910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:15.885313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.886804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:15.889639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 186/370] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:15.890519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:15.977021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:15.979433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.023706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeead76-63fa-46ab-9e27-5ff9502993bb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.067652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.068621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.068905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.069422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.070571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:16.070913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.072292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.084185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 187/371] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:16.085550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.125729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 185/372] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:16.126733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.140387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.143051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.218678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4950d4-02bc-4432-ac83-4a5f52b16c95 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.246845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e976b30-0153-4075-b562-9b2e13b4add7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.263415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.264612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.265327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.265799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.267169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:16.267679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.270465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.278811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 188/373] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:16.280280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.287973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.288645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.289055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.289346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.424177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54974be2-776c-45ef-82e9-7f83f2ef67e4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.435135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.471201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.474664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.474664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.474664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.474664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.475172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.478026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 189/374] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:16.479989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.481479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.483180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.504013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.622041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227f4f8-e5cb-45e5-a049-2d94fd150faf None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.635476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 186/375] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:16.637343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.674744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.674744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.674744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.674744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.675313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.677882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.680367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 190/376] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:16.685675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.755165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.812268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38edd9-2347-47ba-8bf2-efb4eee012f3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.831040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:16.856502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.857369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.862749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.862749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] PATH_INFO: `/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.862749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] RBAC: Authorizing `identity:get_project(project_id=77e7e794d9844bf98e02bc9b4f6dae9f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.862749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:16.863028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 187/377] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:16.871158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:16.873461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4665e27-210e-4bc1-806a-159d62466d93 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.983440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:16.984107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:16.984381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:16.984662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:16.985493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:16.985810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:16.993202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70d3e7-123a-457b-ae46-29bd7eebec23 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:16.995449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:17.005885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 191/378] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=417b7981c02647a98c1d113122c9bdad => generated 260 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:17.031982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:17.032859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:17.033148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:17.033417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] PATH_INFO: `/v3/groups/3d25e2f08ca947619f62f1007023c6c9` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:17.040465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] RBAC: Authorizing `identity:get_group(group_id=3d25e2f08ca947619f62f1007023c6c9)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:17.042114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9ac71-b42d-459d-b6af-54bf8bf44536 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:17.044360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 188/379] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:16 2022] GET /identity/v3/groups/3d25e2f08ca947619f62f1007023c6c9 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:17.052368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:17.202540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef64ca-456a-46b0-91e3-7021a61221dd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:17.211178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:17.244790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:17.245381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:17.245670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:17.245946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:17.246863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] RBAC: Adding query filter params (group.id=3d25e2f08ca947619f62f1007023c6c9, role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=77e7e794d9844bf98e02bc9b4f6dae9f) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:17.247227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:17.248689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211755-1b9f-4371-8e66-2bf2f8f54202 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:17.268742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 192/380] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:17 2022] GET /identity/v3/role_assignments?group.id=3d25e2f08ca947619f62f1007023c6c9&scope.project.id=77e7e794d9844bf98e02bc9b4f6dae9f&role.id=d38d6cda16454360a4f48025ab4df728 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:17.401812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9befc-7dbb-4084-a1b9-01950a8b563d None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:17.402149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9befc-7dbb-4084-a1b9-01950a8b563d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:17.402368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9befc-7dbb-4084-a1b9-01950a8b563d None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:17.404203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 189/381] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:17.409579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:17.409856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:17.410135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:17.432249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:17.433472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:17.531435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 193/382] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:17.565497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9befc-7dbb-4084-a1b9-01950a8b563d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:17.627355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:17.708870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9befc-7dbb-4084-a1b9-01950a8b563d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:17.745876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:17.746759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:17.747371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:17.747803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:17.759229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:17.767781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:17.768853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:17.769875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:17.772315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 190/383] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:17.779710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:17.989772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:17.992637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac1b8e-bb35-4280-8754-2327b4d0fc51 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:18.036868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:18.037540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.037907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.038172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.038941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:18.039248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:18.040628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:18.054379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 194/384] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:18.060123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:18.214531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:18.218093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18382a39-7657-4b71-879b-adcdc7280a98 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:18.260044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:18.260820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.261105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.261379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.262555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:18.264039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:18.265950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 191/385] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:18 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:18.271936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:18.385136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aac31-be3b-4492-8f94-d5ba7ab0f978 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:18.420432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:18.421541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.421993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.422411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.423796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:18.425379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7f995-f18a-43e7-88e3-2b5bfcd82332 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:18.427588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 195/386] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:18 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:18.433359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:18.479146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7bab-3ef4-43ec-8d64-896a3d89a4f8 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.479772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7bab-3ef4-43ec-8d64-896a3d89a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.480254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7bab-3ef4-43ec-8d64-896a3d89a4f8 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.482224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 196/387] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:18.489804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.490428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.491125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.510350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:18.512187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:18.541997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:18.595439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 197/388] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:18.596310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543bf75-787d-4e33-b66e-ac71e93da771 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:18.631171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:18.649992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:18.652077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.652794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.654042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.658491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, group_id=2125ec70f1dd44f2973c3abdf61300f3, role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:18.660445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:18.738449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 192/389] 10.222.0.74 () {58 vars in 1442 bytes} [Wed May 25 14:35:18 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/417b7981c02647a98c1d113122c9bdad => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:18.772187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78887f11-d83f-461e-b9eb-217ec9e80ef9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:18.816555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:18.818317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:18.818867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:18.819345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:18.834594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:18.843361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:18.844236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:18.845039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:18.847399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 198/390] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:18.853396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:18.888770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:18.986561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796ca4e-802b-475b-a857-5db5490ceb9d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.035122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.036090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.036539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.036971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.037953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:19.038427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:19.040148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890cf1e-244f-4eff-b303-ebdbb3aff568 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:19.054336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 193/391] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:19.060478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.089309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:19.187299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89200764-c216-4d1b-9f9c-13587c1f7242 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.234562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.235485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.235776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.236061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.237371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:19.239434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:19.239984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540080d-4d0b-4f85-ad4c-28f67ac7fc01 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.240505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540080d-4d0b-4f85-ad4c-28f67ac7fc01 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.240990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540080d-4d0b-4f85-ad4c-28f67ac7fc01 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.241432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 199/392] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:19 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:19.242709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 194/393] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:19.247337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.248363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.248883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.249490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.276210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:19.277870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:19.394648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 195/394] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:19.409037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:19.420287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.422628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eba3a1-1c8f-41f5-aa76-3cc02a9d57ee None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.458402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:19.463678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.464726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.465209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.465628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.467189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:19.468848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:19.471071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 200/395] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:19 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:19.476973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.570226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5d1c7-3d85-4fc0-9a57-43b719d72716 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.654199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.654959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.655248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.655530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.675853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:19.685260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceb8f9-7e78-4f9f-883d-2bb4bf968429 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:19.690941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:19.692114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:19.693102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 196/396] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:19.697858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.767948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.768852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.769142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.769429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.776428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ae2485352046b2a02d4a6d037cf10e, group_id=2125ec70f1dd44f2973c3abdf61300f3, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:19.778925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbb57b-fc33-42d0-b2f5-89bbe19b09d1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:19.827350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:19.836234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398ea3-2939-4875-8f62-6b7aefa9d1ab None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:19.872960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 201/397] 10.222.0.74 () {58 vars in 1442 bytes} [Wed May 25 14:35:19 2022] PUT /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:19.875215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc646-71d1-4a6e-a598-2d8386ea9d8b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.875524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc646-71d1-4a6e-a598-2d8386ea9d8b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.875797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bc646-71d1-4a6e-a598-2d8386ea9d8b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.877048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:19.882467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 202/398] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:19.883090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.883090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.883090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.883090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] RBAC: Adding query filter params (name=member) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:19.883090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:19.884479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bb00-5200-40c6-9aca-4bdc5c5c3c5c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:19.884772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bb00-5200-40c6-9aca-4bdc5c5c3c5c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:19.885053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3bb00-5200-40c6-9aca-4bdc5c5c3c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:19.886747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:19.897056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 197/399] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:19.902330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:19.914334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7ba3bb00-5200-40c6-9aca-4bdc5c5c3c5c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:19.914334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7ba3bb00-5200-40c6-9aca-4bdc5c5c3c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:19.928741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.026663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 203/400] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:20.026663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe489-7a69-4ae3-82b5-7f548d82dac3 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.026663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe489-7a69-4ae3-82b5-7f548d82dac3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.026663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe489-7a69-4ae3-82b5-7f548d82dac3 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.027457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 204/401] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:20.028906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.029198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.029476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.048671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:20.051484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:20.074327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ab9d7-4f0b-481c-b7bb-6dc59eca856c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.122107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.123271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.123564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.123889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.125781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.127924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.130408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 198/402] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:19 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.131289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.147037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.155724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 205/403] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:20.160642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.253785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.267260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057588fc-9709-4c14-8bc0-6218bf5c6dff None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.285750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cebdc-103c-49ec-a3bb-aaee0633ddf3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.311196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.312248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.312664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.313176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.326297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:20.329616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.330573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.330889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.331166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.332924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.334351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.336559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 206/404] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.337100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:20.337919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:20.338825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:20.340046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 199/405] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:20.347570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.448281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.465345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78ae7-21a4-48c3-a2bc-01a6ee16b1e5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.480894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4998fa-9419-4a3d-bbd6-73410ef5eee4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.502164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.503120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.503407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.503681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.515412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:20.518726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.519615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.520680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.521171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.522115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:20.522624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:20.523857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:20.524728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:20.526112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 207/406] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:20.526859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.527528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.530144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.537270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 200/407] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.538479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.633185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003c620-c006-40f1-b5f8-e10125bb888a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.641291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f4b3-777d-45e3-ba26-134e683dba5f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.668279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.669176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.669443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.669722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.672043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.672627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, group_id=2125ec70f1dd44f2973c3abdf61300f3, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.674982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.677795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.678563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.678972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.679557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.680320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:20.680711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.682126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.690667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 201/408] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.692283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.751073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 208/409] 10.222.0.74 () {58 vars in 1442 bytes} [Wed May 25 14:35:20 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:20.752808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.821810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d1b55-475d-426f-9477-739ff44bb2c6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.827440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:20.869427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.870400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.870720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.871003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.871882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d198-0b37-47e8-98af-89204c6e9b1c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:20.872246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.873854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.876674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 202/410] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.883614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.913112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:20.914331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:20.914849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:20.915321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] PATH_INFO: `/v3/groups/3d25e2f08ca947619f62f1007023c6c9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:20.916996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] RBAC: Authorizing `identity:get_group(group_id=3d25e2f08ca947619f62f1007023c6c9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:20.918727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:20.920922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 209/411] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/groups/3d25e2f08ca947619f62f1007023c6c9 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:20.931767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:20.961738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:21.010349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ae628-5452-4ac8-a20d-ed11f6696241 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:21.054059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:21.055113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.055724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.056015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.057486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:21.059139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:21.061543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 203/412] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:21.067156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:21.109259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660ede7-6ddf-4982-830c-34df4b658325 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:21.147585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:21.153557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:21.154415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.154732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.155081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] PATH_INFO: `/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.156342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] RBAC: Authorizing `identity:get_project(project_id=77e7e794d9844bf98e02bc9b4f6dae9f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:21.157932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:21.160176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 210/413] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:20 2022] GET /identity/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:21.167030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:21.235801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2572e-2240-41b9-9d67-4089b5f475e0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:21.267328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:21.274224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:21.275200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.275491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.275815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.278298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3d21ed6b344bd69f818ae093ef6727, group_id=2125ec70f1dd44f2973c3abdf61300f3, role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:21.279787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355c505-2f6b-40f1-a085-07800aed8951 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:21.329250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c70fb5-1f48-4416-afac-213185e15721 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:21.368020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 204/414] 10.222.0.74 () {58 vars in 1442 bytes} [Wed May 25 14:35:21 2022] PUT /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727/groups/2125ec70f1dd44f2973c3abdf61300f3/roles/417b7981c02647a98c1d113122c9bdad => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:35:21.374029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:21.382547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.382547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.382547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] PATH_INFO: `/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f/groups/3d25e2f08ca947619f62f1007023c6c9/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.382547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] RBAC: Authorizing `identity:create_grant(project_id=77e7e794d9844bf98e02bc9b4f6dae9f, group_id=3d25e2f08ca947619f62f1007023c6c9, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:21.382547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c73d6-d79e-4955-8fe8-d5bd2fb80b57 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:21.478113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 211/415] 10.222.0.74 () {58 vars in 1442 bytes} [Wed May 25 14:35:21 2022] PUT /identity/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f/groups/3d25e2f08ca947619f62f1007023c6c9/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:21.608276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf3f8-2570-4a9d-b8a6-ca8299dee819 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.608596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf3f8-2570-4a9d-b8a6-ca8299dee819 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.608844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf3f8-2570-4a9d-b8a6-ca8299dee819 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.610400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 205/416] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:21.616330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.616941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.617446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.637630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:21.639201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:21.707458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:21.719636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 212/417] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:21.743919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf3f8-2570-4a9d-b8a6-ca8299dee819 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:21.852072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf3f8-2570-4a9d-b8a6-ca8299dee819 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:21.887465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:21.888701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:21.889179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:21.889647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:21.901238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:21.909786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:21.917331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:21.918116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 206/418] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:21.920748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:22.030794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd054b-5651-4a1c-816e-13753b3a267c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:22.070260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:22.070980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.071263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.071539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.072317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:22.072632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:22.074067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42b8a4-2db6-4f73-a9ea-c1065fbc4498 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:22.095188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 213/419] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:22.101552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:22.195311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:22.251557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453128a-7952-4b74-b4cb-d14982175a93 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:22.308775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:22.309440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.309726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.310010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.410107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:22.411710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:22.421015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c246-c69e-4b4f-9973-1d32093b739b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.421311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c246-c69e-4b4f-9973-1d32093b739b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.421596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c246-c69e-4b4f-9973-1d32093b739b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.423441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 214/420] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:22.428484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.428780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.429069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.463373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:22.464501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:22.487376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:22.540144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 207/421] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:22.545802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:22.573711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:22.607824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 215/422] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:22.642961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:22.720057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a691-8c4c-4c89-adb6-81de67411af4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:22.763687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:22.764554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.764841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.765120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.769540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:22.771577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:22.773296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 208/423] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:22 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:22.779401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:22.822726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:22.847892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798710a-fcfb-412e-9749-76fc72de4ca6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:22.931396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba89f1-3586-41f6-85e2-0e2f9fea7137 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:22.960262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:22.961086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.961374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.961653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.978203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:22.979212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:22.979495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:22.979773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:22.983231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:22.986805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:22.989405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:22.992481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 209/424] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:22 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:22.997856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:23.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:23.003030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:23.003833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:23.009769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 216/425] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:23.011138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.060239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.130398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaaff2-b647-4c14-baa1-902590609606 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.143497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.153118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392204-51bf-476c-9181-2a9161e7c407 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.189580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.190548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.190885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.191173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.192061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=50ae2485352046b2a02d4a6d037cf10e) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:23.192385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:23.193830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:23.200793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 210/426] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:22 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=417b7981c02647a98c1d113122c9bdad => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:23.207551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.208261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.208550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.208837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.209645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:23.209963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:23.216070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018797bb-42bf-47bb-81f6-c003f0e52bbc None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:23.231307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 217/427] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:23.237390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.396330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7d25f-2760-419a-b09c-d718751cd51e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.416303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76faec-cf03-4558-ba5c-1c79a886915b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.416600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76faec-cf03-4558-ba5c-1c79a886915b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.416878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76faec-cf03-4558-ba5c-1c79a886915b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.418452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 218/428] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:23.423904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.424195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.424475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.457341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:23.459703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:23.464741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.466129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.466507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.466876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.492806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.554540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:23.561162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:23.561318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 219/429] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:23.588111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.625561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.687339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 211/430] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:23.695199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.703502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28fb6e-3507-48bf-a72c-c6477265d380 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.741936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.743200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.743682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.744151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.755067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:23.760360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:23.763652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:23.764513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:35:23.765299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:23.772904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 220/431] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:23.774337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.828125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58dd3-22d8-4162-9cc7-3980ac4a0b59 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.874142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:23.877814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.878709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.879542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.879809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] PATH_INFO: `/v3/projects/50ae2485352046b2a02d4a6d037cf10e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.881193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] RBAC: Authorizing `identity:get_project(project_id=50ae2485352046b2a02d4a6d037cf10e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:23.882951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:23.889556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810bb-a269-4154-8178-5ee4a5b56b7f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:23.889756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 212/432] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/projects/50ae2485352046b2a02d4a6d037cf10e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:23.890760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:23.928867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:23.929967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:23.930446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:23.930939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:23.931909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:23.932411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:23.933980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:23.948070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 221/433] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:23.949256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.002277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d2c9-90c6-492e-84fc-58dee813b47e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.021250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.038800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.039742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.040025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.040352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.041704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.043199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.049498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 213/434] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.050546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.066985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce797125-b213-47d2-84e7-de21d41ec0e0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.113855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.114509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.114828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.115108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.143299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.167851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04378a3e-528e-456d-b845-f6890e510dd9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.195508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.196980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.205663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.206312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.206591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.206912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.208311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=50ae2485352046b2a02d4a6d037cf10e) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:24.208625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.210293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e753c-cc04-438c-86cc-af33d67e367d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.216555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 214/435] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=50ae2485352046b2a02d4a6d037cf10e&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.217738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2a1e-19a0-42b8-b247-f3ba8f8f5b7d None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.218023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2a1e-19a0-42b8-b247-f3ba8f8f5b7d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.218300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2a1e-19a0-42b8-b247-f3ba8f8f5b7d None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.219711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 215/436] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:24.225310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42aae-69b4-4b43-84a8-08eb6efb425c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.225862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42aae-69b4-4b43-84a8-08eb6efb425c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.226321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42aae-69b4-4b43-84a8-08eb6efb425c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.244625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9cb42aae-69b4-4b43-84a8-08eb6efb425c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:24.246235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9cb42aae-69b4-4b43-84a8-08eb6efb425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:24.304771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.335344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 216/437] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:24.344375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767cd2-5636-4214-a132-870963613311 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.344561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767cd2-5636-4214-a132-870963613311 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.344802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767cd2-5636-4214-a132-870963613311 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.346522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 217/438] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:24.355224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.355224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.355224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.355520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 222/439] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:24.357539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.373642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:24.374901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:24.430558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.458539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 218/440] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:24.459638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.490290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd4e4-6d97-42fb-a8ed-4a5d4792af1b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.530155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.531077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.531362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.531638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.542218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:24.550688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:24.552112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:24.553088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:24.554716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 223/441] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:24.556199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.577413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd973b-e2b8-49b3-ba6b-0facc852c8ed None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.589628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.619470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.620565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.620688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.621229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.623502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.625085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.627331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 219/442] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.628172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.678297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268ddeb-a353-4173-9507-d75ddcedb6fe None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.699233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.721199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.722102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.722393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.723229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.735710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:24.744159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:24.745061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:24.745831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:24.746471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e123185-898e-41e0-84df-c99f9469078f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.746555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 224/443] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:24.746555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.788536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.789353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.789646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.789923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.790699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:24.791822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.793198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.803571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 220/444] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.804552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.845332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.856877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9be5bd-3e95-4011-959b-8f875a0dc80b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.891087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.892139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.892426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.892707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.893892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.895567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.897180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 225/445] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.897947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.903529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fa70f-2059-4f07-a882-972d9c5196af None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:24.937714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:24.938379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:24.938690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:24.938976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:24.939721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:24.940036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:24.941392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:24.944492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:24.949239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 221/446] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:24.956499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:24.999912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800efc9-81dd-42ee-b44b-4f392912bd5a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.033121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.033782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.034050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.034332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.048339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ffbf1-9e45-48b1-9221-f1ef3f704cc4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.049836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.081945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.083138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.083421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.083694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.084522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=3f2e67fc62664875876c5e7f51498b6d, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:25.084853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.086511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.092258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 222/447] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=3f2e67fc62664875876c5e7f51498b6d => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:25.093025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.102521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.104010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.156343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.204956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e65-c27d-4dcf-9fb8-e5e40bdbc8af None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.219170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 226/448] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:25.225574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.248896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.249585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.249871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.250171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.288247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.333127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.334394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.342262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d32c1-a20a-4384-b3a4-5a7c1dfd64c8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.384269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.385059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.385338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.385608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] PATH_INFO: `/v3/projects/4b3d21ed6b344bd69f818ae093ef6727` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.386904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] RBAC: Authorizing `identity:get_project(project_id=4b3d21ed6b344bd69f818ae093ef6727)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.388359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.390047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 227/449] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/projects/4b3d21ed6b344bd69f818ae093ef6727 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:25.396052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.411696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.464247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 223/450] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:25.469815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.507489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641297b-c57c-4f55-9bff-557af3c6d27a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.539344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.540439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.541271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.541549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.541824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] PATH_INFO: `/v3/groups/2125ec70f1dd44f2973c3abdf61300f3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.543017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] RBAC: Authorizing `identity:get_group(group_id=2125ec70f1dd44f2973c3abdf61300f3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.544445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.546047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 228/451] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/groups/2125ec70f1dd44f2973c3abdf61300f3 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:25.551720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.573172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344e73-0c7d-4ea6-a7c5-1267030c603d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.606597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.607471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.607753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.608023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] PATH_INFO: `/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.609233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] RBAC: Authorizing `identity:get_project(project_id=77e7e794d9844bf98e02bc9b4f6dae9f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.610700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.612377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 224/452] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/projects/77e7e794d9844bf98e02bc9b4f6dae9f => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:25.617675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.630866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:25.650018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0478d-b7e5-47f2-bd1c-de6372d47592 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.682991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.683641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.683933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.684216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.685045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] RBAC: Adding query filter params (group.id=2125ec70f1dd44f2973c3abdf61300f3, role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=4b3d21ed6b344bd69f818ae093ef6727) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:25.685370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.686826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.693069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 229/453] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/role_assignments?group.id=2125ec70f1dd44f2973c3abdf61300f3&scope.project.id=4b3d21ed6b344bd69f818ae093ef6727&role.id=417b7981c02647a98c1d113122c9bdad => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:25.717219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3752d-6631-4d19-97d7-d110b4ece0c6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.751946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.752821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.753108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.753385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] PATH_INFO: `/v3/groups/3d25e2f08ca947619f62f1007023c6c9` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.755233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] RBAC: Authorizing `identity:get_group(group_id=3d25e2f08ca947619f62f1007023c6c9)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.756691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7e6a3-fba7-4db4-8e77-56430cb689b6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.758395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 225/454] 10.222.0.74 () {56 vars in 1180 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/groups/3d25e2f08ca947619f62f1007023c6c9 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:25.764514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:25.867573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7ed30-cc4b-46c2-916f-5e12e1ae3f6a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:25.903749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:25.904327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:25.904614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:25.904888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:25.905713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] RBAC: Adding query filter params (group.id=3d25e2f08ca947619f62f1007023c6c9, role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=77e7e794d9844bf98e02bc9b4f6dae9f) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:25.906025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:25.907512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ad68f-2faf-43c0-9c96-70995553f7ff None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:25.913361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 230/455] 10.222.0.74 () {56 vars in 1376 bytes} [Wed May 25 14:35:25 2022] GET /identity/v3/role_assignments?group.id=3d25e2f08ca947619f62f1007023c6c9&scope.project.id=77e7e794d9844bf98e02bc9b4f6dae9f&role.id=d38d6cda16454360a4f48025ab4df728 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:28.597334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4f1-d721-4692-93be-de50de35e2bb None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:28.598715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4f1-d721-4692-93be-de50de35e2bb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:28.599160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4f1-d721-4692-93be-de50de35e2bb None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:28.600962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 226/456] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:28.605986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8c540-d875-4fbf-9452-4befea8aba63 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:28.606402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8c540-d875-4fbf-9452-4befea8aba63 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:28.606547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8c540-d875-4fbf-9452-4befea8aba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:28.628441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1fd8c540-d875-4fbf-9452-4befea8aba63 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:28.629949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1fd8c540-d875-4fbf-9452-4befea8aba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:28.632245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:28.711439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 231/457] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:28.764065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b4f1-d721-4692-93be-de50de35e2bb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:28.767292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:28.909445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b4f1-d721-4692-93be-de50de35e2bb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:28.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca60bf-c101-462a-92af-f72d08a6f393 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:28.929757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca60bf-c101-462a-92af-f72d08a6f393 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:28.930257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca60bf-c101-462a-92af-f72d08a6f393 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:28.933275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 232/458] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:28.937629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1763-9529-4b8b-9fd0-4b3db4d3cd99 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:28.938145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1763-9529-4b8b-9fd0-4b3db4d3cd99 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:28.938625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1763-9529-4b8b-9fd0-4b3db4d3cd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:28.956080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-db0e1763-9529-4b8b-9fd0-4b3db4d3cd99 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:28.958134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-db0e1763-9529-4b8b-9fd0-4b3db4d3cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:28.959344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:28.960397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:28.960679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:28.960951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:28.975132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:28.976576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:28.997753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:28.998672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:29.000190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:29.000190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:29.000190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:29.000190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:29.000190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 227/459] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:29.008625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.058233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 233/460] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:29.064499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264df707-7eda-4260-bafa-1c11a85ef56e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.065022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264df707-7eda-4260-bafa-1c11a85ef56e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.065589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.066728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264df707-7eda-4260-bafa-1c11a85ef56e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.068675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 234/461] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:29.073545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.074095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.074648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.103003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:29.104551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:29.123681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ebdd-1659-40eb-b71c-66a6c8f197f9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.161547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.162591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.163237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.163856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.164768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:29.165365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.167224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.183081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 228/462] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:29.184533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.190946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.202823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 235/463] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:29.204217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.324609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5298e-493c-4cdb-92d9-373de2ac7418 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.340850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207e38a-4031-46e7-93b9-0e9f2f6c2457 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.359339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.368332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.369375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.370139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.370582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.371448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.373039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb8a1-2ecb-41d3-ac6b-0f28a31616a6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.381415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.382456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.383443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.384236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.396153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.397703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:29.416541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:29.417448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:29.418289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:29.418289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:29.418289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:29.418289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:29.418289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 229/464] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:29.418546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 236/465] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:29 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ad8d1-5375-451b-9866-cf509b060c86 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ad8d1-5375-451b-9866-cf509b060c86 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ad8d1-5375-451b-9866-cf509b060c86 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 237/466] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:29.474714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ad8d1-5375-451b-9866-cf509b060c86 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.537355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0af04-43eb-42f3-bfaf-70a25e5481c3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.565625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.585424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.586489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.586963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.587399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.602699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.605390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.620052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ad8d1-5375-451b-9866-cf509b060c86 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-da1706fd-15e6-44c0-819e-68c0939045c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:29.633995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:29.635120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:29.636149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:29.636149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:29.636149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:29.636149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:29.638786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 230/467] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:29.640615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.641331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.641804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.668084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:29.669521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:29.671635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.672326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.672611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.673694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.674495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:29.674856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.676559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.685134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.698040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 238/468] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:29.701479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.745670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 231/469] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:29.746330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.813627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f7ddb-c0a6-43dd-9dfc-f2dd38ff042d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.829138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:29.854993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.855668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.855957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.856241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.856975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:29.857587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.859381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.860563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7feae6-b144-40a2-a6f8-c1110fb392dd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:29.871453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 239/470] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:29.872278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:29.905113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:29.905759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:29.906044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:29.912631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:29.912631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:29.912631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:29.975257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b09e1-f281-464c-ac12-f9994bdfa740 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.009985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.010810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.011100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.011381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.022467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:30.023967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-fd85d95e-07da-4d25-a3c9-c59a2bd5ddf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:30.041697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:30.042720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:30.043627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:30.043627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:30.043627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:30.043627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.043627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 240/471] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:30.044539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b8121-6bac-4d40-a7e5-9a78fa2b252b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.044830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b8121-6bac-4d40-a7e5-9a78fa2b252b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.045110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b8121-6bac-4d40-a7e5-9a78fa2b252b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.050556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 241/472] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:30.050556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b8121-6bac-4d40-a7e5-9a78fa2b252b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.064385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 232/473] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:29 2022] POST /identity/v3/users => generated 248 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:30.065451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.075115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:30.096518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:30.185613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092f270-4ca6-4416-abaf-8d446d236062 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.199355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b8121-6bac-4d40-a7e5-9a78fa2b252b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.225571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.226249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.226716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.227081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.227940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:30.228320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:30.229878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649fe-dd8e-4cb4-a375-f3e176a0f729 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:30.246459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 233/474] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:30.247898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.248266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.248625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.249423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.250281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.250736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.251210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.252065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:30.254313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:30.268878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:30.270411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:30.337080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:30.346033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 234/475] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:30.347119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.362786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 242/476] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:30 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:30.369748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.529546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:30.547010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caa37e-932b-4347-8e65-e889e09d1ce1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.580457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613aca6-e573-4821-b778-7536ff9062dd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.588401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.589088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.589378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.589661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.590431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:30.591911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:30.630016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.sql.core [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:35:30.630016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:35:30.630016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [parameters: {'id': 'da328fa462a342dfa5eeec53c17f86c8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:35:30.630016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92306) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:35:30.635810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.636480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.636762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.637039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.637804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:30.639326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application conn.query(q) May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application result.read() May 25 14:35:30.649281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return ref.to_dict() May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:35:30.650156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application next(self.gen) May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application yield resource May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application next(self.gen) May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise self.value May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application session.commit() May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:35:30.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self.session.flush() May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application compat.raise_( May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:35:30.651938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:35:30.652799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application conn.query(q) May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application result.read() May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:35:30.653621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [parameters: {'id': 'da328fa462a342dfa5eeec53c17f86c8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:30.654394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:35:30.663624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.665657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 243/477] 10.222.0.74 () {60 vars in 1131 bytes} [Wed May 25 14:35:30 2022] POST /identity/v3/roles => generated 144 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:30.665657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.740284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 235/478] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:30 2022] POST /identity/v3/users => generated 249 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:30.777548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c692ef-ca63-499e-ae37-e7c6a6da96b0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:30.818864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:30.819621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:30.819908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:30.820190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:30.833081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-09a1d276-1a24-45b0-b96d-7fb9ec40f4d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:30.841455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:30.842339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:30.843172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 244/479] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:35:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:30.850004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:30.939114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:31.000155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165f027-7120-47fc-886b-30f4e5f78b09 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:31.099953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:31.100542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:31.100824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:31.101098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:31.101845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:31.102152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:31.103538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c25ac-a96b-4838-8aa7-f212078877b5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:31.117841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 236/480] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:31.950414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec69a-d3cc-41ff-a3c2-72537739e15d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:31.951491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec69a-d3cc-41ff-a3c2-72537739e15d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:31.951984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec69a-d3cc-41ff-a3c2-72537739e15d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:31.953800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 245/481] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:31.964355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:31.965234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:31.965714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:31.987295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:31.988520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:32.065242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 237/482] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:32.096224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec69a-d3cc-41ff-a3c2-72537739e15d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.106866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:32.222811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec69a-d3cc-41ff-a3c2-72537739e15d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:32.261693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:32.262747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.263205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.263632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.283936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:32.292029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:32.293092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:32.294076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:32.300194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 246/483] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:32.307744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.378382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:32.447926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a121-07de-45fa-921a-9fb2fdc3d6ed None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:32.501819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:32.502409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.502714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.502993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.504279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:32.504593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:32.505958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9f4a-64da-4df4-a20a-4bebc0156e18 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:32.514604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 238/484] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:32.523723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.604954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:32.648331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbf308-e1f2-4ce5-a68c-a381fd4edb64 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:32.696697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:32.697494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.697776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.698047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.698235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8ee7-a52e-41dd-9837-9c13355e87b7 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.698527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8ee7-a52e-41dd-9837-9c13355e87b7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.699944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8ee7-a52e-41dd-9837-9c13355e87b7 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.701090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 239/485] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:32.711168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:32.712917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:32.715368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.715664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.715948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:32.731486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:32.732338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:32.733163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:32.733163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:32.733163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:32.733163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:32.735382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 247/486] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:32.747288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.768109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:32.769107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:32.852836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d75f3-63d9-4497-a4c9-90086254aaf9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:32.887015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:32.893649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:32.894608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:32.895148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:32.895659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:32.896586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:32.897094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:32.898692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:32.912220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 248/487] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:32.916241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 240/488] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:32.927280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.942200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:32.984733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.064367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee17467-6c6e-407b-b79f-98f2cc10d667 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.101398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.102580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.103204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.103681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.116260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:33.124803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:33.125692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:33.126505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:33.129361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 241/489] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:33.135714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.173094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb34b8-511f-4f5b-a5b2-7aa0dadda935 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.253591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.254440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.254750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.255033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.264607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.266082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:33.302148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:33.303385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:33.304274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 249/490] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:35:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:33.313723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.314107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799eb97-0918-42cd-bee3-3df991ba2fb6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.346998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.353224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.361317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.381036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 242/491] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:33.381865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.402189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.475502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a292d6-f7a3-4b03-a0e2-62969e30782a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.495614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3d5d5-ead4-460f-8849-cfae564eec7d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.512187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.513534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.514027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.514506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.515588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:33.516531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.518546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe733cb8-d8cd-4135-b63f-b7a8ca92cf49 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.530038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.530861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.531144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.531425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.542224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.543717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-e5b25f5e-e374-49a7-a68c-5d9f035a8b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:33.561435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:33.562467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:33.563608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:33.563608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:33.563608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:33.563608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:33.567244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 243/493] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:33.567375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 250/493] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:33.568590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87530b-493d-4349-9005-4d4580168836 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.568880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87530b-493d-4349-9005-4d4580168836 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.569146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28fbc4-02d5-4ffd-adf6-3a2044dd28cc None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.569241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87530b-493d-4349-9005-4d4580168836 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.569632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28fbc4-02d5-4ffd-adf6-3a2044dd28cc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.570048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28fbc4-02d5-4ffd-adf6-3a2044dd28cc None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.570693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 244/494] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:33.571402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87530b-493d-4349-9005-4d4580168836 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.571838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 251/495] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:33.575638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.576112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.576564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.587777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.595349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:33.596788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:33.667157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 252/496] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:33.668418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.695100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87530b-493d-4349-9005-4d4580168836 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.696120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.731233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.731896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.732179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.732456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.733178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:33.733485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.734894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31583b3d-2923-4f77-8232-8a2907f9b109 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.746736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 245/497] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:33.748044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.748338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.748618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.768316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:33.770246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:33.778582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5e160-64a3-4779-a097-c3c455de130a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.814097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.814878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.815056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.815322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:33.835991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.853162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 246/498] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:33.853162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:33.898572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:33.900415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:33.928044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:33.955149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6efca-67f9-47a2-924a-77b2f9add623 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:33.991241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:33.992029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:33.992311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:33.992586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.003048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:34.011400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 253/499] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:34.011400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:34.011790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.012648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:34.013418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 247/500] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.014547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.084528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.122476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2042b45-1baa-4da9-a063-7b9828f7bcf9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.130402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2402-5caa-4cdf-8011-6b576cc4d3b7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.148689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.158103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.158927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.159213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.159488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.166331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.167145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.167334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.167432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.167689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.168794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.178758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:34.183654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.185008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:34.186068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:34.186703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.188059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 248/501] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.190123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.195614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 254/502] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.196819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.291201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddafb2-1a9a-4130-a475-9d76e610b93f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.295485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d045e-2b2d-4154-b00a-7083e7621963 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.308539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.328947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.329785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.330166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.330405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.331812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624366f-7642-44e2-a383-028468595c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.332473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624366f-7642-44e2-a383-028468595c22 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.332760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624366f-7642-44e2-a383-028468595c22 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.333038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624366f-7642-44e2-a383-028468595c22 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.333787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624366f-7642-44e2-a383-028468595c22 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:34.334100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624366f-7642-44e2-a383-028468595c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.335783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624366f-7642-44e2-a383-028468595c22 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.341338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.342811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.343799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 255/503] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:34.344588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624366f-7642-44e2-a383-028468595c22 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:34.360029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.360896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.361735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:34.361735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:34.361735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.361735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:34.361735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 249/504] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.361998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.395736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.445752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624366f-7642-44e2-a383-028468595c22 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.461516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1124-bc03-4e00-aeeb-c3900bb2f682 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.479555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.480172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.480462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.480743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.481507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:34.481807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.483233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f10a1-8a66-4c10-b0ea-b011ea1cbd76 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.491978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 256/505] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:34.493230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7e45c-b427-4694-ae63-221ded48e9b0 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.493488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7e45c-b427-4694-ae63-221ded48e9b0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.493788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7e45c-b427-4694-ae63-221ded48e9b0 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.495030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 257/506] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:34.495726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7e45c-b427-4694-ae63-221ded48e9b0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.498147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.498910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.499207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.499491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.500335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:34.500651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.502103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.538582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 250/507] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:34.539579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.567830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.596185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7e45c-b427-4694-ae63-221ded48e9b0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.606816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.629582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.630373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.630688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.630973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.638847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea2ec8-7c83-46e3-8598-619b2db6f2ad None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.642000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.643477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:34.660896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.661864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 258/508] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.677516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.678167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.678449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.678753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.679470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:34.679777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.681136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440d1b8-e34d-4ffb-8ce1-2ab868296f6e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.691950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 251/509] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:34.693089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.693374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.693647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.708859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:34.710303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:34.764463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e66d95-179b-4353-b61e-7d0253997dc5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.775550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 252/510] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:34.776978 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.798974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.799552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.799836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.800113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.806161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.810947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.812410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:34.829335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:34.830168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:34.831043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:34.831043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:34.831043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:34.831043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:34.835201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 259/511] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:34.835875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.852207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:34.886550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8841378-6f9f-48ba-960c-b370f0a41156 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.920335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.920904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.921184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.921459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.933140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8787e-e80c-4c38-ac62-490c5ad42b14 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:34.968406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:34.969063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:34.969347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:34.969630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:34.970348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:34.970681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.972066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:34.983629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 260/512] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:34.984425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:34.991360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:34.992802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.010088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.059814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.085354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810608-bdbf-4170-b5a9-6e9eeda6004a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.090659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 253/513] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:35.091494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.120504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.121375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.121662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.121944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.129470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.130963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:35.145592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:35.146469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:35.147356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 261/514] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:35.147811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.190851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363ae1e-341d-4f35-b3d9-17025d901ad3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.207419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.224258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.225033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.225314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.225589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.235972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.237423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.249989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f7f2-ca2f-48b8-afde-3038cbf1e43b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:35.255416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:35.256407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:35.257449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:35.257449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:35.257449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:35.257449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:35.257449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 254/515] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:35.258174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.285560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.286211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.286494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.286813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.287530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.287841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.289200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.290572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.301428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 262/516] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.302052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.359439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b6bd1-092f-493e-8852-5fb05ceb5dec None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.392699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.393477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.393759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.394035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.401332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.402222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4cf92-9497-435b-9d33-609beb1c0bad None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.402812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:35.418668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:35.419536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 255/517] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:35.421033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.439011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.439803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.440088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.440364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.450865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.451261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.452289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:35.469163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:35.470017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:35.470886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:35.470886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:35.470886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:35.470886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:35.470886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 263/518] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:35.471541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.516802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.524253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85452f5e-eae4-4bb9-93bd-c5fe0090ccf6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.557693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.558352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.558656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.558944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.559734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.560047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.561485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.569460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67560-a6e2-4bd7-8942-42e774256e01 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.578878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 256/519] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.579498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.607542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.608192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.608474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.608756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.609470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.609780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.611189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.622577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 264/520] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.623395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.668819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.678652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43d37-55d3-46f6-baee-3044d9c62765 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.712141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.712870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.713152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.713427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.721247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.722029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ccab3-df31-41bc-a9ec-24a562266cba None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.723276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.729896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:35.737343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:35.743147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 257/521] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:35.744116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.760848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.761657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.761955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.762519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.763385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.763711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.765125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.782779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 265/522] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.783584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.843156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30364c67-fda9-4901-87f2-ca466a48e134 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.876373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.877041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.877327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.877613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.878331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.879208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.879980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d005c-b8dc-4545-a376-c370b7b43cf5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:35.880621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.892191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 258/523] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.892947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.904712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:35.918387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:35.919065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:35.919352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:35.919632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:35.920463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:35.920776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:35.922201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:35.927729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 266/524] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:35.928492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:35.961855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.008831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd6cb-738d-4170-b9a3-dcf28ba36f0f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.043119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025731e1-fe5e-4df5-8c52-c23bdf8bd824 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.048912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.049668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.049954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.050287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.051214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.052704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.079156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.083041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.083041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.083041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.083041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:36.083041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.083562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.102398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 267/525] 10.222.0.74 () {56 vars in 1139 bytes} [Wed May 25 14:35:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:36.103294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.146121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 259/526] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:35 2022] POST /identity/v3/users => generated 247 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:36.147428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.196967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.214189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d50993-0e66-488d-b06a-5991a5442bd5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.251233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.262003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.263040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.263500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.263917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.291378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239cfc-7425-4ca4-a44f-624973460242 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.346295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.347830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.348565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.348855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.359521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.364008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.366877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.368480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:36.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:36.384063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:36.384905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:36.387388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 260/527] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:36.388388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.492239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.506317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902e5fe-1d31-4e6d-bfa1-eb32337fa53b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.518004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 268/528] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:36.519312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.544404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.557078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.558680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.559316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.559316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.636607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.638082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.650002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ba9e5-6932-430e-8f95-24033387a911 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.691828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.692499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.692787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.693066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.693860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:36.694180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.695672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.714350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 269/529] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:36.715245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.770893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 261/530] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:36.771687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.775886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.800378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:36.834664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeb7af-a830-4b11-88ca-b7d2c83bd11c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.873125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.873926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.874212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.874488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.874844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312e37-63a4-4ef3-9b7f-daa8ffca7315 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:36.886343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.887976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:36.906860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:36.908725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 270/531] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:36.911205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:36.911926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=6cdeab1ae06e4df9a270a716ae0a1db3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:36.918876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:36.922101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 262/532] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/role_assignments?user.id=6cdeab1ae06e4df9a270a716ae0a1db3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:36.923008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.021642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.024710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.029024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eae73e-fc88-426a-9a77-d8429169aefb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.031893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c03d64-eb6e-4550-9c75-b67d581ebdc1 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.067293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.069010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.069396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.069774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.070723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.071414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.071700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.071983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.072742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:37.073060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.074536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.082534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.084082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.089358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 263/533] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:37.094401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:37.106906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:37.109002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 271/534] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:37.111143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.213856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10a8af-f843-4a74-8c8b-ddfbdfa32562 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.222099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613522e-2d64-44e1-93d6-bfb3eddaa64e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.258837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.259175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.259583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.259770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.259917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.260057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.260194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.260331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.261061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:37.261386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.262978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.268760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.271208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.277756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 272/535] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:37.288169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:37.299659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:37.300551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:37.301378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 264/536] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:37.308038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.319017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.334195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.420586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a3f35-a749-436a-9561-e8d2d082bdf8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.424635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2d8f-e3c5-49a3-98ff-a9e833deda37 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.480800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.482736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.483616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.484570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.488985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.489873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.491122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.494775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.495799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:37.496319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.498103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.501016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.502758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.524917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 265/537] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:37.530276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:37.537905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:37.538812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:37.539654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:37.544198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 273/538] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:37.549418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:37.650683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.676319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799eb8-3208-40f6-902d-c712d1b600a0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.712705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.713809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.714308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.714855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.715900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=ffea1035a1d2434e948256087a112b4b) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:37.716419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.718064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.721711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:37.726345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 266/539] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/role_assignments?user.id=ffea1035a1d2434e948256087a112b4b&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:37.777873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c714d-ff70-4ca9-88ab-dd91a60d9537 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:37.834110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:37.834992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:37.835284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:37.835564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:37.836364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:37.836681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:37.838131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a61a8-bbcd-4bf2-a663-372d1e7ee003 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:37.856012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 274/540] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:37.862895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:38.064568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc440f-08d6-40c6-be7a-d16ecb91c9fc None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:38.101849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:38.102429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.102739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.103021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.103925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=931dba704d244ccdbb9f7dbadc68d7df) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:38.104244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:38.105748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6e6bc-a977-4ed1-8b3c-29cbc62e5343 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:38.107709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:38.112157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 267/541] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:37 2022] GET /identity/v3/role_assignments?user.id=931dba704d244ccdbb9f7dbadc68d7df&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:38.262969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c5ab-bb26-49ba-805a-f9db1c238ba8 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.263482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c5ab-bb26-49ba-805a-f9db1c238ba8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.263555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c5ab-bb26-49ba-805a-f9db1c238ba8 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.265155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 275/542] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:38.270932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.271235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.271500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.300417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:38.302224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:38.412157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 268/543] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:38.435598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903c5ab-bb26-49ba-805a-f9db1c238ba8 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:38.565300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903c5ab-bb26-49ba-805a-f9db1c238ba8 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:38.603379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:38.610311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:38.611385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.617190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.617190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.627508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:38.636058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:38.637091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:38.638039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:38.644334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 276/544] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:38.655727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:38.809284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ba225-bb4c-4a1d-82a8-89f3763d3041 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:38.848967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:38.860206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:38.860879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.861168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.861451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.862206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:38.862515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:38.864487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194652d-6afa-40ea-aef8-c4afd84f7201 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:38.878305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 269/545] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:38.884538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:38.906038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae377ab-dc9e-4a61-9bc6-7be35abe2933 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.906328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae377ab-dc9e-4a61-9bc6-7be35abe2933 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.906610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae377ab-dc9e-4a61-9bc6-7be35abe2933 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.908280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 270/546] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:38.914111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:38.914404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:38.914714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:38.941158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:38.942352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:39.062830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 271/547] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:39.073682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3d3a-19bf-4438-a9e8-90b9d6307a4b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.094093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.132520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.133653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.134066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.134480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.160371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.161903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:39.200941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:39.202022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:39.202991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:39.202991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:39.202991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:39.202991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:39.202991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 277/548] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:39.211715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.215202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5fbe9-d1eb-4ab4-b9aa-79b1fea71061 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.240464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:39.263543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.264639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.265101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.265562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.291849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:39.302665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:39.303160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:39.304171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:39.305050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:39.307274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 272/549] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:39.308946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.425085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00b23-52d2-47b8-96dd-447723ff13f7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.465177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d1f83-0c52-48ff-9aaa-9cf184c4f4c4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.502552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.503524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.504046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.504428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.505258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:39.505681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.507295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.511508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.512190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.512475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.512755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.513519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:39.513833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.515224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.525366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 278/550] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:39.526590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.544822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 273/551] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:39.567625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.645864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae829a-e046-481d-9bab-ecbf3c38e3ba None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.647946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:39.688668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.689834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.690128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.690412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.697331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:39.699593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.701078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:39.718348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:39.719519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:39.720606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:39.725515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 279/552] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:39.729850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.818542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31dcf-a1d7-4cf8-9a01-324d84d03dc7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.869428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.871059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.871587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.872410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.883499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b088c-e887-4714-b0c6-0945984682e1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:39.888258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.889961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-334f22cf-22cc-4dd9-a416-4cf025d1f8ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:39.913829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:39.914784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:39.915664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:39.915664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:39.915664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:39.915664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:39.918719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 274/553] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:39.920240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d232-30d7-48ea-b2ad-f6bf0e784eca None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.920733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d232-30d7-48ea-b2ad-f6bf0e784eca None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.921229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d232-30d7-48ea-b2ad-f6bf0e784eca None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.926738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:39.926738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.926738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.926738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.927077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:39.927077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:39.927403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 275/554] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:39.928434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991642d2-9272-4941-9034-85d4af9a4f9c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:39.928553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d232-30d7-48ea-b2ad-f6bf0e784eca None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:39.954765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 280/555] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:39.956473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:39.957713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:39.958002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:39.978974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:39.980540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:40.010959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.035023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.043945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d232-30d7-48ea-b2ad-f6bf0e784eca None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.070667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 281/556] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:40.070667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.103717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.105077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.105569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.106050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.107043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:40.107592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.109181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.127352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 276/557] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:40.130050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.192858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1529ab-209c-4f1a-8c00-2c617f62c1f5 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.236365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faf986-e538-458d-9305-4695a307630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.237166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf986-e538-458d-9305-4695a307630a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.237454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf986-e538-458d-9305-4695a307630a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.237737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf986-e538-458d-9305-4695a307630a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.253170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faf986-e538-458d-9305-4695a307630a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.255267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faf986-e538-458d-9305-4695a307630a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.257365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569302-8719-4fda-a8a7-4bef36051943 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-47faf986-e538-458d-9305-4695a307630a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:40.273198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:40.274612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:40.276044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:40.276044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:40.276044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:40.276044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:40.279426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 282/558] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:40.280383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faf986-e538-458d-9305-4695a307630a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.290971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.304649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.305750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.306452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.307139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.321081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:40.330039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:40.331252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:40.332709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:40.333667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 277/559] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:40.334273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.354154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.395794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faf986-e538-458d-9305-4695a307630a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.431735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.432538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.432829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.433112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] PATH_INFO: `/v3/users/swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.440879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.442607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.444333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c96f33-20a7-4c0e-bb06-9eb3aae084a0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f8c9397b-19a8-4d37-baee-7fdccc8a32af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:40.458180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:40.459393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:40.460853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:40.463284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 283/560] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:40.464618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa65b-17ca-4751-b2f3-eb844924287c None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.464916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa65b-17ca-4751-b2f3-eb844924287c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.465201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa65b-17ca-4751-b2f3-eb844924287c None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.466599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 284/561] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:40.467556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa65b-17ca-4751-b2f3-eb844924287c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.490533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.491309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.491598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.491881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.492606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:40.492921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.494319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.505796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 278/562] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:40.506605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.546783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.585097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.590248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa65b-17ca-4751-b2f3-eb844924287c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.611252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a649d-6e64-46dd-a187-9ff8747e17f6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.627044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.627743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.628034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.628319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.629136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:40.629454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.630917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d748d-96e6-4642-8908-96dc01909ba5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.644692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 285/563] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:40.644692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.644692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.644692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.657284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.657946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.658235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.658515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.659390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:40.660268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.661698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.662860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:40.664068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:40.684880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 279/564] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:40.684880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.742678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 286/565] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:40.742678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.801417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3662f-24d3-457c-92d8-a58388af793c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.801871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.831977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:40.837091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.837913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.838192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.838466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.844994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.846145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6618c0-7ae8-46cb-9c32-f7a99645f4c3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:40.846513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:40.860404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:40.861287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:40.862175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 280/566] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:40.862691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:40.885917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:40.886740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:40.887031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:40.887311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:40.897945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:40.899434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:40.917448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:40.918354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:40.919501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:40.919501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:40.919501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:40.919501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:40.919501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 287/567] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:40.920145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.041794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240313c-b867-483a-8316-d72d1b4b41a4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.074011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.074673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.074964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.075240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.107038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513072-2d89-4aff-bbcb-0962483a38ae None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.114123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.134668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.140898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.141680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.141965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.142238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.143864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.145957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.154092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-89da2bc5-7864-4558-9f04-6679ab526335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:41.161899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:41.163042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:41.163889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 288/568] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:41.164818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbfb3-b250-4ff6-a475-1520b3347f4a None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.164818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbfb3-b250-4ff6-a475-1520b3347f4a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.164818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbfb3-b250-4ff6-a475-1520b3347f4a None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.166050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 289/569] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:41.166805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfbfb3-b250-4ff6-a475-1520b3347f4a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.250949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 281/570] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:41.251790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.266108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfbfb3-b250-4ff6-a475-1520b3347f4a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.299767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.300352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.300685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.300961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.301734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:41.302043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.303534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.317539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 290/571] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:41.318355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.345128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.347334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.355459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b1185-f842-429d-99e6-18dae6eb650c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.389301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.389954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.390237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.390511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.391263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:41.391577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.393017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cae737-59b4-4cda-a255-22273e0b0e53 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.404883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 282/572] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:41.406548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.407237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.407640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.416964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb52f-fa93-4a19-9c90-459e4a6fb3a0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.423495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:41.424674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:41.450797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.451457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.451742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.452028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.452769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:41.453082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.454440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.463014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 291/573] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:41.463795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.493588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 283/574] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:41.494383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.555287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.567245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775969c-1b72-495a-95d3-9f62b7a6233d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.577919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.595281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2354-f653-4398-94fb-25a4b01ed1cf None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.599536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.600515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.601124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.601403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.611285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.612717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.630051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.630953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.631254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:41.631397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:41.632425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:41.633716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:41.633716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:41.633716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:41.633716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:41.634027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.634113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 292/575] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:41.634113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.647627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=d4beeef3c39c44f39c8c7ee5f13e9623, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.649071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.730052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117fae3-145a-4835-bbef-5756ef3cb361 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.747535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 284/576] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:41 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:41.748332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.770096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.770981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.771263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.771544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.779773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.781322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.785447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:41.799724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:41.801478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:41.802534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 293/577] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:41.803163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.883918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d195f9-48b6-4a2c-9c68-0cd12f81751d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.917099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:41.919738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c39158-b104-45ff-8ed3-6fcd24f32cf8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:41.923331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.924135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.924898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.924898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.940714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:41.942316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:41.960872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:41.961097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:41.961918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:41.962857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:41.962857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:41.962857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:41.962857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:41.963097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:41.963097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:41.963097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:41.963283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 285/578] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:41.963891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:41.978173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:41.986039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:41.987207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 294/579] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:41.989799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.093993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.137205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b6e2e-a291-4252-9e70-b9f31a0dffda None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.144740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc478e-bcfa-47e3-9d9f-b7c2bc9ad2f2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.173866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.174545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.174856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.175139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.176763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.177083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.178520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.181203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.181867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.182155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.182425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.183179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.183496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.184847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.199874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 295/580] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.199874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.206752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 286/581] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.206752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.256445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.301188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726504-3b8f-4f40-bac4-6b99b1e60eda None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.318090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009326d9-e2ca-4c1f-988e-d5b70f23dbb6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.323334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.348892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.349555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.349843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.350125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.350891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.351206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.352588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.356379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.357040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.357324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.357604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.358334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.358659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.360068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.369871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 296/582] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.369871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.370104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 287/583] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.370104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.468069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85452ad-05ce-45ad-a2ca-8ad9373c1e63 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.476324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.478116 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a86-e324-493c-8883-7d5f0e484456 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.503420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.504216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.504501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.504774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.511351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.512796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.514004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.515015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.515461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.515891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:42.527601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:42.528567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:42.530145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:42.530593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.530886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 288/584] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:42.530974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.532116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:42.549954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:42.550974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:42.551840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:42.551840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:42.551840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:42.551840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:42.551840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 297/585] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:42.552119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.589191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.651049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58befc2e-89d3-4e92-a963-8fa6e87fb5e4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.667143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173e0f8-aa46-4dc9-968a-48cc6e639d3a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.685502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.686284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.686566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.686875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.695579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.697021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.701997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.702828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.703118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.703402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:42.712078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:42.712982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:42.713898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 289/586] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:42.714473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.719738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.721151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:42.739481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:42.740380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:42.741248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:42.741248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:42.741248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:42.741248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:42.743547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.745390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 298/587] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:42.746017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.818833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145bb0-6674-4833-9a24-16c4693b7b94 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.827380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.847182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402efa8d-1ad3-4cde-8937-322c2bca0c7d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:42.854216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.854930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.855220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.855499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.856314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.856747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.858121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.874983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 290/588] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.874983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.891322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:42.892478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:42.892728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:42.892950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:42.893686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:42.893998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:42.896028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:42.908122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 299/589] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:42.913224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:42.972706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:42.990411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff935b-9a45-4f92-a2d9-07a9b1edad41 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.018425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2286ae-d47c-4892-aa16-18e25669e249 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.025824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.026495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.026807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.027083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.027909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.028218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.029622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.047336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 291/590] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.052942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.055127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.055787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.056085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.056977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.057087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.057345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.058730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.071775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 300/591] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.072460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.075165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.166210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cb642-4a2c-4f3d-8ed5-22fad3e73d82 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.182337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffcd6-366d-49fe-8bcf-0c29549beaed None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.201305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.201952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.202236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.202510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.203372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=66a7eff73b85406a8874547263816a68) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.203688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.205120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.211191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 292/592] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/role_assignments?user.id=66a7eff73b85406a8874547263816a68&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.211839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.229557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.232236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.233144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.233432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.233714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.242806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.244316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:43.260704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:43.261674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:43.262549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:43.267021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 301/593] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:43.267990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.348990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.358416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d23082-f308-4a34-a88d-be36c7adeedf None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.394809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ea87f-3037-4700-9a53-c8565fd0eb0d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.406859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.408235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.408726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.409360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.424037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.426131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.434922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.435857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.436221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.436501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:43.444230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:43.445135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:43.446454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:43.446454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:43.446454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:43.446454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:43.446716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.446789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 293/594] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:43.446871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.446942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:43.468638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:43.469647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:43.470612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:43.478167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 302/595] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:43.479886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.508612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.549942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884d5e-9a8e-47a7-9640-72ffc2c3298d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.586269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.586988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.587273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.587551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.588340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.588653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.589817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56598b9-d9ee-4bbe-a38c-f8e12ad9ad0c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.590074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.607944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 294/596] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.608854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.629212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.630290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.630837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.631408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.632485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.633004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.634887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.652694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.653058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 303/597] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.654163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.713126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a62e03-0acb-4b24-89d7-9cad68f7e84c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.734764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:43.749767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.750435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.750858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.760408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.760408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:43.760408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.760408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.773397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 295/598] 10.222.0.74 () {56 vars in 1139 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:43.774399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.782628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33e49c-0b82-427a-8c82-680364752600 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.823967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.825103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.825456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.825809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/6cdeab1ae06e4df9a270a716ae0a1db3/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.843395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=6cdeab1ae06e4df9a270a716ae0a1db3, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.844858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.888908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed62d5-cd4a-40cc-b200-3b9711227cb5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:43.930957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 304/599] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:43 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/6cdeab1ae06e4df9a270a716ae0a1db3/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:43.932014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:43.932141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:43.933141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.933620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.934108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.951799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:43.953625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-2917b7c7-ab20-4421-83c7-ac74c8126dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:43.986006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:43.986957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:43.987882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:43.990694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 296/600] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:43.992491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee56d50-a4ed-49c4-8e9b-edbdc016b436 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:43.992941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee56d50-a4ed-49c4-8e9b-edbdc016b436 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:43.994543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee56d50-a4ed-49c4-8e9b-edbdc016b436 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:43.995726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 297/601] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:43.996733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee56d50-a4ed-49c4-8e9b-edbdc016b436 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.000145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.055897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.078079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06906e9-7653-4082-b6dc-273eaf39538f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.119283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.120103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.120391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.120673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.132183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.133670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.141377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee56d50-a4ed-49c4-8e9b-edbdc016b436 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e3be36b0-f631-41aa-a8ef-30ee68148be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:44.153362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:44.154279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:44.155187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:44.155187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:44.155187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:44.155187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:44.159034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 305/602] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:44.160409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.160705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.160992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.179884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:44.181358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:44.186063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.186939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.187303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.187694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.188615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:44.189001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.190557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.205163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 298/603] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:44.206198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.274034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 306/604] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:44.275461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.299770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.338278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.341426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aac33-493d-406b-a663-240bed15a6d2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.377996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.378827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.379176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.379533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.380341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:44.380727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.382177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.398425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 299/605] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:44.398425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.416574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fef81-b10f-45d0-b36b-6cd8715b1ce6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.468132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.469280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.469718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.470404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/ffea1035a1d2434e948256087a112b4b/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.492832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=ffea1035a1d2434e948256087a112b4b, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.495643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.507119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce456d43-1a0f-446f-9374-3423d348c531 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.544489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.545252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.545534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.545854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.557485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:44.565269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:44.566402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 300/606] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:44.567273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.583124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 307/607] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:44 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/ffea1035a1d2434e948256087a112b4b/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:44.584182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.613049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.724244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:44.795706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7b191-79c6-42cb-b4b6-89283aa95c28 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.799925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdb136-2314-46cd-86cb-215cfda4d82b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:44.834944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.835900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.836338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.836626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.838348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:44.839157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:44.839542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:44.839830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:44.840581 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:44.840894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.842264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.844031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:44.845992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:44.852476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 308/608] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:44.864042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:44.864234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:44.865125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:44.866010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 301/609] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:44.871724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.001242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:45.034930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8169-6c00-46ed-84c6-2f71b045dbd6 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.044464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbba697-5140-4272-bc81-f2151f5e703e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.095093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:45.108478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.109286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.109570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.109846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.119320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.126117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.139239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.140717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.152730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 302/610] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:45.157463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:45.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:45.168052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:45.168985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:45.168985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:45.168985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:45.168985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:45.174909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 309/611] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:45.179777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.348939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bc7ca-9b10-42ed-8a94-23d95aa2ce7d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.368423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86952b9-5fac-4962-ac70-d80d12feabdc None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.398716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.399645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.399932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.400214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/931dba704d244ccdbb9f7dbadc68d7df/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.412491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.413195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.413550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.413845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.414600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:45.414956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.416420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.431013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:45.432163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 310/612] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:45.437723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.441848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=931dba704d244ccdbb9f7dbadc68d7df, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.445091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.498063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:45.548520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 303/613] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:45 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/931dba704d244ccdbb9f7dbadc68d7df/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:35:45.556749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cb462-49ea-4735-bab2-539efc261007 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.595713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.596512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.596802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.597145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.606167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.607606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c87aa17b-ca9f-4716-a89c-8ab3f1d0f5c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:45.623199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:45.624239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:45.625189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 311/614] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:35:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:45.635866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.681564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec1d4-65f1-46f8-b6f4-7004c43b3662 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.681903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec1d4-65f1-46f8-b6f4-7004c43b3662 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.682147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec1d4-65f1-46f8-b6f4-7004c43b3662 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.683589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 312/615] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:35:45.689584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.689904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.690153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.721859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:45.723174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:45.815409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f777ad2-214e-4c8a-8069-911d41ffb10f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:45.850352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 313/616] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:35:45.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:45.857953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:45.858241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:45.858521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:45.859404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:45.859719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:45.861158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:45.874524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.897897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 304/617] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:45.903443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:45.924465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.015309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.016391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743b268-4cf0-4ca0-a548-627ada14d39e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.056889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.057699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.057991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.058277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.070129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:46.079301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:46.080450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:46.081288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:46.081972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 314/618] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:35:46.090501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.203375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa0dd3-12ad-4bd5-937c-d75a9069e0e8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.204685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c337f-5a7d-4b3e-9e0f-09b182101e41 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.207112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.245356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.246081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.246880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.247171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.248083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:46.248407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:46.249797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:46.260338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 315/619] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:35:46.268309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.297881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.298531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.298847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.299117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.388532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:46.390151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd32ce-94e1-4bfe-a36d-c306ab2362be None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:46.443309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.482912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad54c82-be17-4c4d-a40f-647b80459196 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.512338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 305/620] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:46.513860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8216dab-cebc-41d5-a064-0ca298da8ab0 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.514232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8216dab-cebc-41d5-a064-0ca298da8ab0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.514663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8216dab-cebc-41d5-a064-0ca298da8ab0 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.516403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 306/621] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:46.517350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8216dab-cebc-41d5-a064-0ca298da8ab0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.527902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.528710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.528990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.529267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.540563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:46.542022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c763c17a-0654-44c1-b1bf-3c6ba2fbedf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:46.561718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:46.563853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:46.566047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:46.566047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:46.566047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:46.566047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:46.571267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 316/622] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:46.571267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.571267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.571267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.587814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:46.589093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:46.607015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.657119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8216dab-cebc-41d5-a064-0ca298da8ab0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.666986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 317/623] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:46.667939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.697419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.698235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.699045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.699332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.714228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:46.716027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:46.726654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:46.742117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:46.743050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:46.744587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:46.744587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:46.744587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:46.744587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:46.744587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 307/624] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:46.745260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.817277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3362a-fdfd-4a00-9f39-86583d9584eb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.851780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:46.857225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.858000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.858357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.858782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.859615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:46.859992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:46.860613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415cb3-b90b-42eb-aa59-2f2c17f4d609 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:46.861503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:46.878403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 318/625] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:46.879550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:46.905803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:46.907567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:46.908076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:46.908800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:46.925308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:46.936899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:46.937940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:46.938817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:46.945000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 308/626] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:46.945992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.004297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50fcd-cb78-4b8f-8ab2-197e806b0535 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.016782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.040920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.041940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.042483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.043039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.043978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:47.044504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.046078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.059021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 319/627] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:47.060209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.064497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432da1f-45dc-4b26-ade0-8b866d66b3d0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.102793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.103597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.103887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.104170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.113420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.114989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.120676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:47.131888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:47.138197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 309/628] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:47.139307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.173276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d954-1e7b-4b0b-9926-14ea066c09cf None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.210145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.211122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.211596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.212059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.212997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:47.213498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.215088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bc471-f5fc-4012-b363-f44b2f69fa14 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.223564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 320/629] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:47.225077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c4b15-51bc-4587-9c0c-218890502ecc None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.225370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c4b15-51bc-4587-9c0c-218890502ecc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.225654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c4b15-51bc-4587-9c0c-218890502ecc None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.227082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 321/630] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:47.227765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c4b15-51bc-4587-9c0c-218890502ecc None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.251088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1731e6-3106-47a6-a026-b60ba2e744f5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.270613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.287447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.288329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.288604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.288892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.295538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.296997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:47.312158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:47.313066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:47.313939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:47.318426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 310/631] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:47.319355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.346572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c4b15-51bc-4587-9c0c-218890502ecc None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.361121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.381625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.382300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.382591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.382905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.383746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:47.384065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.385507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18161bed-db2d-4325-bbf0-abd1c091fced None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.404108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 322/632] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:47.405351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.405644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.405930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.422180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:47.423501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:47.436592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3024-bb23-4cfe-af7c-399dbd367c8a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.478763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.479556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.479842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.480117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.490312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.492298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.506672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 323/633] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:47.506672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:47.518839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:47.520465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:47.521502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:47.521502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:47.521502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:47.521502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:47.521502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 311/634] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:47.522470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.554704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.622732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a227-734a-4a45-9c7a-bbaa793a1b97 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.625128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.629319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd59b-c166-4ab6-b2df-6b30470b50a5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.664994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.665237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.665658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.665827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.665983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.666086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.666251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.666353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.667085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:47.667381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.668831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.677632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.679191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.685905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 324/635] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:47.685905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:47.702728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:47.703677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:47.704575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:47.704575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:47.704575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:47.704575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:47.704575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 312/636] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:47.709203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.792894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.794276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67780de1-4c76-4a16-ad8a-7af45a1bb3a8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.811205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b793-b224-4f99-9c40-89668a1a3ece None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.830036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.830985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.831514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.832082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.832827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:47.833140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:47.834855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:47.847355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:47.848179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:47.848466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:47.848748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:47.856451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 325/637] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:47.856451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.866857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:47.878182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:47.879975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:47.880806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 313/638] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:47.882179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:47.883909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:47.966502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5bf0e-ddb4-4b25-ab1e-523f538988a8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:47.992181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae13701-aeed-44b3-a2fa-703efc52fefb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.002818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.003514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.003806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.004108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.004922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.005242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.006704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.022675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 326/639] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.022675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.041043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.041757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.042051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.042329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.043184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.043445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.044864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.058787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 314/640] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.059669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.062823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.131895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.148123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc960d3-6611-4b64-bed9-bc2e134b72b1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.174137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b813f5b-87de-4555-a89e-52d6c2fa24a0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.189168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.190081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.190373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.190662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.198548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.200037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.214992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-cf4e0f18-35c7-436b-95ff-e290e249bcb7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:48.215255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:48.216251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:35:48.217583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:48.218133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.218133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.218133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.218336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 327/641] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:48.218403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.218403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.218975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380068d2-eb4f-496f-a1ad-b314fce6b436 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.218975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380068d2-eb4f-496f-a1ad-b314fce6b436 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.218975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380068d2-eb4f-496f-a1ad-b314fce6b436 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.219167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.225089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 328/642] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:48.225089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380068d2-eb4f-496f-a1ad-b314fce6b436 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.234452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 315/643] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.235143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.311387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.345298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380068d2-eb4f-496f-a1ad-b314fce6b436 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.355963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c4f30-b67a-4c12-99d6-c920531f7ac2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.382274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.383066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.383541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.383653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.390907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.391155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.391569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.391851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.392128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.392613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.392988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.393261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.395214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-759df175-a708-4b0b-b1e9-7235388da5ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:48.407204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:48.408139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:48.410949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 329/644] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:48.413242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.413867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.414214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.418247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 316/645] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.419134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.427149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.433736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:48.437105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:48.504811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 330/646] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:48.505799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.530820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1776a3-facf-4e37-86eb-9a2ae0a4ab7b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.555301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.568546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.569210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.569492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.569764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.582056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.583853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:48.602647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:48.603610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:48.604524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:48.604524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:48.604524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:48.604524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:48.608463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 317/647] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:48.609042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.622805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514df9e-d4ab-4bdf-9d63-c73312a9a3da None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.660337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.660993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.661298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.661614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.662327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.662652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.664086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.667098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.676268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 331/648] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.677299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.720351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daaff5-aa1b-482f-8d1e-1da7a74a2687 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.755964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.756605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.756894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.757174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.781018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2dbd4-bdb2-412d-8467-d4ed28ce59a0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.804469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.817124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.817923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.818209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.818480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.828885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:48.832012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.833475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:48.837300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:48.838190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:48.839406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 332/649] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:48.840105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.895837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:48.937844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53e11c-c21a-4c73-aaf3-da28e078cb18 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:48.944924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 318/650] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:48.945736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:48.977393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:48.978137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:48.978418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:48.978718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:48.979437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:48.979747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:48.981111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:48.993685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 333/651] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:48.994601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.027723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.053107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15150-db6b-42dd-bb3b-1be4a4dbcf85 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.089928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.090796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.091091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.091369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/66a7eff73b85406a8874547263816a68/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.098650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da3d57-e9a2-44b8-958c-ad9007962f30 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.105786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=66a7eff73b85406a8874547263816a68, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.107394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.115728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.133218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.133882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.134171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.134452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.135238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:49.135553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.136924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.144712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 334/652] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:49.145488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.182667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 319/653] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:35:48 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/66a7eff73b85406a8874547263816a68/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:35:49.182667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.287982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718222be-8c9d-4895-8a5f-91c2d2f3b61e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.308362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188bd1b-bd1c-428a-b3d7-0083bd26021f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.342745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.342950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.343040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.343040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.343040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.351975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.353019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.353309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.353592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.353779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.355487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.362857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.364564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:49.376652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:49.377637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:49.379072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:49.379072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:49.379072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:49.379072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:49.379072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 335/654] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:49.379794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:49.387000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:49.389138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 320/655] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:49.391396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.423040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.504400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780a0e0-ead9-4244-98db-db60bde36ddf None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.511742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a793-401a-483e-a400-aba231dc1273 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.540485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.541149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.541460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.541741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.542476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:49.542823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.544345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.555571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.556408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.556745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.556871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 321/656] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:49.557057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.557870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.571715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.573621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:49.599045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:49.600087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:49.600977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:49.600977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:49.600977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:49.600977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:49.600977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 336/657] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:49.601416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.609116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.667425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f172076-7910-4ec0-8695-46c656272a45 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.694868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.703568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.704249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.704630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.704915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.705840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:49.706158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.708231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.714753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f158-cfe1-457e-bdf6-ce54a1c9a154 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.727296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 322/658] 10.222.0.74 () {56 vars in 1137 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:49.728260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.754271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.755580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.755986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.756268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.765730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.767228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:49.785564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:49.787433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 337/659] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:49.789203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.833556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0d0e6-0d0e-41aa-929e-b4497df672de None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.863511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:49.872166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.872907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.873249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.873586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.879064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:49.879064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:49.879064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:49.893561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 323/660] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:49.894853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:49.915030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462c40-b5ad-4d7c-97b0-b7834898ee2e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:49.955907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:49.956731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:49.957087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:49.957440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:49.971759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.034255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.035357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31ee81-3dba-4399-bd63-be65a3ca1198 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.035976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.072446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.073160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.073539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.074134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.074781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:50.075207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.076870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.090217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 324/661] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:50.091315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.116750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.169237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 338/662] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:50.170325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.217371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea039f-ff25-4bf5-b849-bdf4c71c0f6f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.228224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.255576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.256473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.256815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.257175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.265844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.267397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:50.283112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:50.284185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:50.285661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 325/663] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:50.286263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.310182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393b00c-f72e-4191-8920-5de1e966ae1c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.356348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.357044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.357330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.357606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.358434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=6cdeab1ae06e4df9a270a716ae0a1db3) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:50.358781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.360220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.368079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 339/664] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/role_assignments?user.id=6cdeab1ae06e4df9a270a716ae0a1db3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:50.368869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.386686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.413010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9427d7-ce9d-4177-8474-07dca01eb9c2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.454493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.455674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.455957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.456234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.470165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.472746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-07bc3aa5-bc9b-44b4-b904-7e96392bc83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:50.499354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:50.500337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:50.501250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:50.501250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:50.501250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:50.501250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:50.501480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3c415-cc1b-4e03-906a-67fa4c1b572a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.503109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 326/665] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:50.504212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610db18-d7b3-4daf-a6ac-19739200a9a4 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.504501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610db18-d7b3-4daf-a6ac-19739200a9a4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.504814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610db18-d7b3-4daf-a6ac-19739200a9a4 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.506126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 327/666] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:50.506866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610db18-d7b3-4daf-a6ac-19739200a9a4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.515500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.547327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.548411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.548923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.549420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.550539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:50.551106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.552805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe324f-6a69-4c71-bec4-33028dd63fa1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.573513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 340/667] 10.222.0.74 () {56 vars in 1139 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:50.575430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.575956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.576459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.595359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:50.597613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:50.624052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610db18-d7b3-4daf-a6ac-19739200a9a4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.662727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.663666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.663956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.664236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.664967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:50.665281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.667241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.668810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.680529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 328/668] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:50.681252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.685561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 341/669] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:50.686453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.797751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.810679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738bbe7-b2e4-4cf8-873b-0ca10a9a2a62 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.817111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8faef-2d65-4173-9d76-cd140a2a6cbf None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:50.855546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.856520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.856900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.857279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.862316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:50.863193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:50.863919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:50.864368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:50.864742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.866324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:50.881927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:50.883057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:50.884143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:50.890318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 342/670] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:50.891293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:50.940091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:50.948531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:50.950160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.011013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838598-bd8b-490d-8066-c3b421bd9f9d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.047498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.048515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.048895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.049268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.060506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:51.068864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:51.070001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:51.071048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:51.077908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 343/671] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:51.078928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.088684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 329/672] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:51.089793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.102168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.194776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.201959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554b2-e036-43cf-bdcb-15ff1637b030 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.220297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6623cd0-7fe4-4868-ab76-4e9748496c55 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.239780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.240822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.241177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.241543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.242387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:51.243045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.244609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.259003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 330/673] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:51.259957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.265079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.265928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.266366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.266835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.267797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:51.268261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.271015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.288234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 344/674] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:51.289381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.383250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.400656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeadc46-12bf-4a0b-90f4-069acc9d7c47 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.408584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc7e88-e8a7-4ffa-9a36-d5bc5b4b2a93 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.447861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.448576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.448860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.449139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.450267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.451240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.451522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.451797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.462933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.463525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.464382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.465614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:51.482592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:51.483509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:51.484403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:51.484403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:51.484403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:51.484403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:51.485931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:51.486806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:51.488236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:51.488236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:51.488236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:51.488236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:51.489208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 345/675] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:51.490057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.491138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 331/676] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:51.492128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.504583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.597578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8958e31-aaf3-4c57-a51a-755942146381 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.599892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8940-552a-456b-b819-993f9a9a055d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.626098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.634261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.634874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.635536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.635828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.636059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.636160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.636595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.636927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=ffea1035a1d2434e948256087a112b4b) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:51.637207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.637306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.638170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:51.638677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.638779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac987b7a-f1bb-4d1d-9209-719a5fb7c8f1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.640285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982db8c-21ff-4d98-bdde-9f6722bdd24d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:51.644659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 346/677] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/role_assignments?user.id=ffea1035a1d2434e948256087a112b4b&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:51.646179 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15805fc-0006-4b7b-9256-a4ea76f4e75f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.646623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15805fc-0006-4b7b-9256-a4ea76f4e75f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.647067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15805fc-0006-4b7b-9256-a4ea76f4e75f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.648920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 347/678] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:51.649893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15805fc-0006-4b7b-9256-a4ea76f4e75f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.666847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 332/679] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:51.668742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.669241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.669802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.698839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:51.701283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:51.905368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:51.912859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 333/680] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:51.913701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:51.944914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15805fc-0006-4b7b-9256-a4ea76f4e75f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:51.990500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:51.991433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:51.991890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:51.992346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:51.993263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:51.993714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:51.995779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.011215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 348/681] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:52.015834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.024310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe723a-0cdd-4741-804f-e3ecef43fea7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.071305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.072663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.073343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.074321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.075359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.084114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.086119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:52.104056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:52.105006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:52.105941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:52.108584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 334/682] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:35:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:52.109660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.127927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aabb4c-a444-4ec1-afcb-0d3df7f3b4ca None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.163989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.164904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.165085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.165348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.176074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:52.183061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:52.183991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:52.184878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:52.185705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 349/683] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:52.186399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.228453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789978e-9d7d-41f5-b0dd-e697a3ddae4a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.279565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.280707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.281598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.282086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.293341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.295368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.306079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45cbb4-b584-41cb-9b67-f04986ed3ac5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:52.314967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:52.315929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:52.316799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:52.319558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.325060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 335/684] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:52.325060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.358107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.359352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.360057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.360533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.362217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:52.362777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.364675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.389112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 350/685] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:52.390322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.436989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b2dcb-92ed-46be-96bf-c5c973b62006 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.457190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.476058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.476821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.477182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.477609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.478476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:52.478884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.480460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.490160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 336/686] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:52.491168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.506055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2d95-c31c-47e7-87b8-308d661aed36 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.545731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.546824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.547400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.547852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.548851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:52.549352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.551209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.569190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 351/687] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:52.570368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.601365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.603885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41145594-9852-4ce9-9fc7-6c15524e4196 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.647349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.648056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.648567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.648919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.686997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db234f3-7711-4d00-8a2b-47f691fe705e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.701406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.723028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.724590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.725707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.726714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.727198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.727687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.738250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.739958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:52.760007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:52.760927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:52.761807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:52.761807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:52.761807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:52.761807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:52.764032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 352/688] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:52.765018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.832284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 337/689] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:52.833478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.853022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.873753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634e5e-62be-4f21-91fa-7ec966648f23 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.909807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.910490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.910807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.911084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.911950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=931dba704d244ccdbb9f7dbadc68d7df) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:52.912992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:52.913758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded299a-d9e8-4119-a8f9-038cb7711a6e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 353/690] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/role_assignments?user.id=931dba704d244ccdbb9f7dbadc68d7df&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14089b-6339-44ad-84a9-394220d53020 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14089b-6339-44ad-84a9-394220d53020 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14089b-6339-44ad-84a9-394220d53020 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 354/691] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:52.926675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14089b-6339-44ad-84a9-394220d53020 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:52.932671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:52.955305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf35533-e7a9-4016-9f20-fd92af21a441 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:52.996110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:52.996938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:52.997223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:52.997501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:52.998406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:52.998817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.000423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191f704-1d81-462d-b686-fbc4b3a33932 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.015454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 338/692] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:53.016906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.017199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.017480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.036654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:53.037899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:53.069630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14089b-6339-44ad-84a9-394220d53020 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.121396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.122464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.122931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.123358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.131089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 339/693] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:53.132181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.137999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.139694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.143286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:53.163730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:53.164844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:53.165846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:53.165846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:53.165846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:53.165846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:53.165846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 355/694] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:53.166274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.219434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.259994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7cbec-bb02-4131-b3bc-04ed5a5bdc2f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.294976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c40ab4-113c-43df-bd27-9aa7ac6d0549 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.307814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.308838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.309128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.309406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.319394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.321548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.337624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.338723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.339212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:53.339474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:53.340400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:53.341316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 340/695] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:53.341898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.342817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.358972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.361284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:53.385517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:53.386454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:53.387358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:53.387358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:53.387358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:53.387358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:53.393193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 356/696] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:53.393193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.447059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.470821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32fda0-ff24-4fab-a8bd-973fbfbf20a5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.504924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.508756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed26dcb-22e2-49aa-bcc9-3afeec011e6f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.512544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.512688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.512953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.526344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.526344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:53.526344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.526344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.533777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 341/697] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:53.533777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.560088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.561305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.561769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.562349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.564047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:53.564544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.567218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.591304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 357/698] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:53.592415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.645392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2399cd-648f-4a97-b272-fe5bee999f87 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.682331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.683147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.683505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.683855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.684660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:53.685038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.686473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.705870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 342/699] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:53.705870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.728899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2dab-99e7-41f4-9abe-260cf4d99fd0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.740259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.762830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:53.769722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.771084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.771861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.772613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.780736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.782386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:53.797480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:53.798472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:53.799440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:53.803111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 358/700] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:53.804472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:53.828090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff963930-afe4-4581-bf7b-da66b240becb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.879581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.880868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.881472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.882233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.916596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e889cf-a0b1-4b31-afbd-9b6ed847a8f9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:53.952467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:53.953297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:53.953587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:53.953868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:53.955660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.956363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:53.974466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:53.975960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337389c9-f42f-48e2-90dd-5b1da8a42e7f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.026059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.048553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 359/701] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:35:53 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:54.048553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.083921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.103277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 343/702] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:54.104848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768723e-d86b-44dc-9193-f7fd3fdb48a6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.105136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768723e-d86b-44dc-9193-f7fd3fdb48a6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.105793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768723e-d86b-44dc-9193-f7fd3fdb48a6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.107852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 344/703] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:54.108609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768723e-d86b-44dc-9193-f7fd3fdb48a6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.200075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36b37-611e-4f83-a6c9-7c21478c7331 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.242336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768723e-d86b-44dc-9193-f7fd3fdb48a6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.248433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.248604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.248856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.249133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.249924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:54.250237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.252253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580c9ae-792d-4485-859f-a86a19fa35bf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.266593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 360/704] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:54.267906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.268196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.268472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.289125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.289923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.290210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.290491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.290608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:54.291760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:54.303402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.304244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:54.326207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:54.327178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:54.328051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:54.328051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:54.328051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:54.328051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:54.328051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 345/705] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:54.328328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.336965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.395613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 361/706] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:54.396882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.439038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.444592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d47db-98c8-406e-8f8a-04e5290e56d4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.480878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.481636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.481979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.482341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.483301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:54.483686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.485165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.491360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 346/707] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:54.499578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.519418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb192b7-6586-4ee2-a621-72f4f1b09dca None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.577141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.577836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.578124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.578406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.579209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:54.579527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.580946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.595634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 362/708] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:54.603747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.672499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.722572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:54.739493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c0fe-36b2-4b3d-8e0f-bc1ce43e580f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.771826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435b747-6176-49d7-b35b-89911e8da90a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:54.784815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.786477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.787008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.787554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.788908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.790791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:54.819419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:54.820426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:35:54.821284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:54.824369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 347/709] 10.222.0.74 () {56 vars in 1129 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:54.830024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:54.837574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:54.838842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:54.844072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:54.844591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:54.855676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:54.858066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:35:54.888716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:54.889694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:35:54.890577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:54.900677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 363/710] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:54.905227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:55.031809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f5f2-b907-440f-81e3-8c0117773fe3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.063863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:55.066988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ced7-5f6a-440f-87bf-f149b319a571 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.081752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.082760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.083210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.085416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.085416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:55.085416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.087413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.100260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 364/711] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:55.106817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:55.127782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:55.132101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.132865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.133148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.133423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.134128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:55.134486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.140136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.145947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 348/712] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:35:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:55.151155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:55.235827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064622f-ecfe-4965-b642-b17681e591c0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.309383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.309957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.310242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.310517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.311376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=66a7eff73b85406a8874547263816a68) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:55.311688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.313149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4a8ac-ae79-4d53-b1c4-56c602e776cd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.332030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 365/713] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:35:55 2022] GET /identity/v3/role_assignments?user.id=66a7eff73b85406a8874547263816a68&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:55.344932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a11cb4-36a4-4b71-842c-9d129387c7e7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.383588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.384527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.384983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.385413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.386305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:55.386908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.388451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d282f-7975-4da1-a1b6-ee15f8fdebda None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.394785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4635-b2cd-41ad-9864-5981f8d42fd9 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.395492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4635-b2cd-41ad-9864-5981f8d42fd9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.395948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4635-b2cd-41ad-9864-5981f8d42fd9 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.397051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 349/714] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:35:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:55.398137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 366/715] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:55.404223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.404527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.404800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.439970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:55.441349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:55.538677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:55.572239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 350/716] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:55.595119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd4635-b2cd-41ad-9864-5981f8d42fd9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:55.664130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:55.709918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd4635-b2cd-41ad-9864-5981f8d42fd9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.745111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.746254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.746819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.747271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] PATH_INFO: `/v3/services/network` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.748095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.749634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:55.763896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:55.764870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:35:55.765684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:55.768150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 367/717] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:55 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:55.774056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:55.884932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e7b10-96e3-4cea-b212-4a9d3b73eec6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:55.928372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:55.929343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:55.929803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:55.930253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:55.931187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] RBAC: Adding query filter params (name=network) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:55.931714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:55.933254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca897d-0676-470d-b115-b48957f80b39 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:55.938823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 351/718] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:35:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:55.944527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:56.082549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc33f7-dcda-44ac-9710-5868f2206653 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:56.111562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:56.136324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:56.136923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:56.137207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:56.137482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:56.138186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] RBAC: Adding query filter params (type=network) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:56.138497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:56.139885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304c112-2727-40b4-9be0-0da942196116 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:56.152470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 368/719] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:35:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:56.920304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270247a9-bf63-4864-aefa-c80fc920b177 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:56.920820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270247a9-bf63-4864-aefa-c80fc920b177 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:56.920896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270247a9-bf63-4864-aefa-c80fc920b177 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:56.922565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 352/720] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:56.928151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:56.928445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:56.928725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:56.951936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:56.953127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:57.041841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 369/721] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:57.075064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270247a9-bf63-4864-aefa-c80fc920b177 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:57.158785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:57.216388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270247a9-bf63-4864-aefa-c80fc920b177 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:57.269563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:57.270375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.270670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.270956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.291717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:57.299649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:57.300625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:57.302063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 353/722] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:57.306987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:57.453632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16b544-bb5b-4557-8487-df42779cce33 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:57.468945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:57.489683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:57.490397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.490740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.491032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.491776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:57.492084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:57.493455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a93a-249d-4d81-9f88-4b16a0a6aa96 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:57.501981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 370/723] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:57.507415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:57.604963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326cca4-d47c-4211-be10-fb2fdc449e33 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.605291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326cca4-d47c-4211-be10-fb2fdc449e33 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.605530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326cca4-d47c-4211-be10-fb2fdc449e33 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.607179 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 371/724] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:57.612630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1babcb-01c5-43d0-af37-fdd1a6d14a25 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.612918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1babcb-01c5-43d0-af37-fdd1a6d14a25 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.613191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1babcb-01c5-43d0-af37-fdd1a6d14a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.651059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1f1babcb-01c5-43d0-af37-fdd1a6d14a25 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:57.652248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1f1babcb-01c5-43d0-af37-fdd1a6d14a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:57.663816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011b756-ee17-48b9-81e6-8ed81c5afc5c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:57.683065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:57.718247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:57.719166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.719392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.719668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.746453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:57.747968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a17a5e67-5fd7-4eb4-a715-b19a0eb1a9e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:57.783025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:57.784069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:57.785035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:57.785035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:57.785035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:57.785035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:57.787322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 354/725] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:57.788812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684ce6-cbd2-419b-81b1-b7c5c474872c None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.789308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684ce6-cbd2-419b-81b1-b7c5c474872c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.789849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684ce6-cbd2-419b-81b1-b7c5c474872c None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.795484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 355/726] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:57.796542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684ce6-cbd2-419b-81b1-b7c5c474872c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:57.797447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 372/727] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:57.798767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.799059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.799339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.818349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:57.819559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:57.905131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 373/728] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:57.907088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:57.944975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684ce6-cbd2-419b-81b1-b7c5c474872c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:57.956796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:57.983207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:57.984133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:57.984606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:57.985053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:57.986001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:57.986482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:57.988120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.001523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 356/729] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:58.002680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.028460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd05f6-d79b-4203-b2b4-a80332d543c1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.032973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.063645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.064445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.064731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.065010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.076289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:58.084760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:58.085835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:58.086712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:58.091007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 374/730] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:58.092079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.130032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3fa0-e7cd-496f-9136-5356e08b7d48 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.174186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.175404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.175725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.176249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.176978 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.178750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2009-ecde-4964-93cc-d64a641e5269 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.214268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.220168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77591dc7-5310-46d2-94e7-469c757a39a4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.255953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.256698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.256988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.257268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.264462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.265933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:58.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:58.286745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 375/731] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:58.287866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.296000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 357/732] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:35:58 2022] POST /identity/v3/services => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:58.308285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca703b-25d1-4f44-b0a7-c69e656243b6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.308590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca703b-25d1-4f44-b0a7-c69e656243b6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.308876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca703b-25d1-4f44-b0a7-c69e656243b6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.310466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 358/733] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:58.311023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca703b-25d1-4f44-b0a7-c69e656243b6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.421287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272af5f-2917-496c-859f-31bf30834737 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.459896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.468966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.469659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.469956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.470588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.471038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:58.471352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.473649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e3d60-90f2-4e22-b6c2-05bc8efea919 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.483652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca703b-25d1-4f44-b0a7-c69e656243b6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.484328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 376/734] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:58.485747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.486162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.486368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.506925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:58.508051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:58.519570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.533318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.535374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.535717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.540665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.540665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:58.540665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.540665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.560455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 359/735] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:58.561326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.608372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 377/736] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:58.614709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.704020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93acf-302b-4cd3-bdb6-e0b38b507d0a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.750200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.751293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.751585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.751869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.756598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa978c-940c-40a3-999d-4efc736d8ea6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.764438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.766250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.769305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:58.792833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:58.798853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 360/737] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:58.799748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:58.802291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.803003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.803289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.803569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.823175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:58.889289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:58.890807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd747cc7-9097-4eb2-96f0-8a25e8fd5a07 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:58.912059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90c7b0-79b9-4c86-b3a2-7bce9fc46af0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:58.947484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:58.948318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:58.948564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:58.948841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:58.960255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:35:58.967985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:58.969111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:58.970014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 361/738] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:58.970954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.030340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 378/739] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:35:59.031748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7a28-d005-401a-b7b6-1d3006e5869c None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.032039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7a28-d005-401a-b7b6-1d3006e5869c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.032321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e7a28-d005-401a-b7b6-1d3006e5869c None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.038040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 379/740] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:35:59.038040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e7a28-d005-401a-b7b6-1d3006e5869c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.059808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.092725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.107021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49b19-af39-4ca0-b1cf-e3b40bb4f743 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.142400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.143111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.143394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.144226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.144952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:59.145264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.146627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.153200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e7a28-d005-401a-b7b6-1d3006e5869c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.163677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 362/741] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:59.163677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.193315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.194000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.194268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.194549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.195328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:59.195641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.197019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926f62-6e0b-4da6-9072-6aae76e25aff None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.207108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 380/742] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:59.208476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.208769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.209048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.224921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:35:59.226112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:35:59.279919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda729a-c415-4ee0-9720-aa67af6a1742 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.294830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.295851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 381/743] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:35:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:35:59.296716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.318543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.319644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.320398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.321129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.333094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.334841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:59.352952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:59.353996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:59.355211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:59.355211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:59.355211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:59.355211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:59.357947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.358910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 363/744] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:59.359424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.427915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845d82f-ea3d-43d5-8fb6-23bc4e064dcf None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.462917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.463707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.463995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.464271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.466267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b5a5d-6793-446b-95b8-7808c5246373 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.472204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.473714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:35:59.490435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:59.491471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:35:59.492409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 382/745] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:59.493607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.509410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.510193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.510479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.510805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.521840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.523312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:35:59.541485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:35:59.542381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:35:59.543368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:35:59.543368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:35:59.543368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:35:59.543368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:35:59.543368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 364/746] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:35:59.543638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.566501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.595231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.601495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f188821-f3dc-489d-b339-4af44b7eca5e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.635943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.636613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.636898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.637177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.637899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.639305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.654282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5765a-4997-438b-8d7d-4220d0d2163c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.688765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.689424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.689707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.689985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.690769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:59.691085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.692453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.704932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 383/747] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:35:59 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:35:59.704932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.705212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 365/748] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:59.705212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.803884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:35:59.828471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa87b4-e82d-41a8-bb0f-93d7a83e2aea None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.861407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080b76-64ca-40e6-b278-bcbc5fdbf452 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:35:59.868255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.869103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.869495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.870022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.870913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:59.871336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.872893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.887849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 366/749] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:59.889100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.906456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:35:59.907639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:35:59.908193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:35:59.908483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:35:59.909604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:35:59.909927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:35:59.911809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:35:59.933153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 384/750] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:35:59.934141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:35:59.953440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.023168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343710b-0aff-4523-bac8-7437db268caa None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.053032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce9cf-d1c6-4325-a884-320070b32835 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.067022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.068103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.068391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.068674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.078305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.080513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.083333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.093165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.095302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.095302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.095302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:00.099519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:00.100586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:00.101692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:00.102890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 367/751] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:00.103900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.106167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.108010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:00.124434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:00.125417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:00.126376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:00.129668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 385/752] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:35:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:00.130760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.224408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971089cd-7b1b-4284-9292-fe46b158cc11 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.229548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.261675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.262859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.263378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.263874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.270892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7721f1-ec65-4ac8-be75-b8bb5d5392e6 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.278974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.280449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:00.306127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:00.307106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:00.308001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:00.308001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:00.308001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:00.308001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:00.308001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 368/753] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:00.308767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.312087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.313018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.313512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.313992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.315027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:00.316192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.317855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.333942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 386/754] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:00.335035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.391568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.455847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68e2e-0d60-4394-b2ac-3d5b8b06f889 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.460500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6ac9-7cdb-42cb-a8b3-152ec54a05e4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.492875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.493697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.494087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.494471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.495449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:00.495817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.495934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.497468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.502723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.504129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.504563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.504989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.505901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:00.513347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.513347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.520722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 387/756] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:00.521685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 369/756] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:00.522280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.522673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.629136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e23cc-bfca-4b20-877c-6a0a1e3c5fff None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.630944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0527c-65f7-4b76-a019-c90d9297df07 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.642741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.665871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.666526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.666842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.667127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.667955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=a0f1db595d804131bf47d7ae396d5889) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:00.668259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.668392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.668910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.669201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.669487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.669696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.686983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 388/757] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/role_assignments?user.id=a0f1db595d804131bf47d7ae396d5889&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:00.686983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.751458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.753035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f69334-aafa-4da1-85cf-365628758fbe None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.771073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.839942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220212-f5fa-4320-a56a-1a7b2ca08e2f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:00.874996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 370/758] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:00.876733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690109-0809-4dde-9e34-66babb2ff482 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.877005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690109-0809-4dde-9e34-66babb2ff482 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.877353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690109-0809-4dde-9e34-66babb2ff482 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.879581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 371/759] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:00.879934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690109-0809-4dde-9e34-66babb2ff482 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:00.891916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:00.893845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:00.895516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.895516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.895516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.907049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:00.907049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b51a123f-6f75-4881-af30-654c9ab74581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:00.919276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:00.920204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:00.921453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 389/760] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:00.922119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:00.922119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:00.922338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:00.945583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:00.946946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:01.030805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 390/761] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:01.031389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690109-0809-4dde-9e34-66babb2ff482 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.031825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.073092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.073898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.074344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.074654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.088983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.090832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.094777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:01.115178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:01.116383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:01.117365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:01.117365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:01.117365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:01.117365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:01.117365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 372/762] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:01.117649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.156308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508013fd-cfdd-460a-9e74-f44555bb1bb7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.183061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.193481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.194389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.194910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.195389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.196352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:01.196866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.198484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.212017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 391/763] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:01.213090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.234941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef01a1-e12f-4018-a440-581b87d67727 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.273449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.274667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.275144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.275810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.276656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.278339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:01.292961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:01.293887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:01.294745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:01.302024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 373/764] 10.222.0.74 () {56 vars in 1129 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:01.303060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.342079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539ed30-385b-48c5-90e1-b9c519f0fd39 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.344500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.385568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.386244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.386540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.387295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.388032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:01.388352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.390131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.406789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 392/765] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:01.407926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.419598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863454c1-0f0d-4f28-859c-192016f608d5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.446579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.458676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.461600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.461600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.461600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.463490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=d4beeef3c39c44f39c8c7ee5f13e9623, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.465343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.526288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a79c-f21c-4bf6-a4ec-de95307dc419 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.540353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 374/766] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:01 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:01.541661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.594061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.594830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.595124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.595407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.596129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:01.596447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.598850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.613340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 393/767] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:01.619832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.676770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.684466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a904f-26a0-4541-ab6a-ac1b892b72fe None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.727801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.728994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.729495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.729974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.740211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.742570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:01.760049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:01.760999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:01.761888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:01.764639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 375/768] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:01.772862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.829559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:01.853050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7cba2-3bd0-4b61-9dbf-e8adfe850dc5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.905609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97f2-7bbc-4b3a-8fa9-abe4cd05ea3b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:01.924458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.925034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.925326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.925611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.926339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:01.926758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.928315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.934338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 394/769] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:01.941919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:01.949714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:01.950605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:01.951221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:01.951649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:01.953064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:01.953533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:01.957971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:01.972325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 376/770] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:01.977814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:02.063681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:02.073657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0942-cad0-46f9-944d-4402015faf9b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:02.112319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:02.113350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.113817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.114355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.115442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] RBAC: Adding query filter params (interface=public, service_id=e660c40908b14bde92289488975bdeed, region_id=RegionOne) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:02.115961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:02.117611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd7a9f-fd65-40e3-bbcd-cb2b6bf9f50b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:02.126086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 395/771] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/endpoints?service_id=e660c40908b14bde92289488975bdeed&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:02.127639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b5-cfd5-4f2b-805f-b8bb9037ac8f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.128624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b5-cfd5-4f2b-805f-b8bb9037ac8f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.129090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d838b5-cfd5-4f2b-805f-b8bb9037ac8f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.130764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 396/772] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:02.132597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d838b5-cfd5-4f2b-805f-b8bb9037ac8f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:02.163083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb8d5c-fedd-48c1-a9fa-77edecda1d96 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:02.166366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:02.204839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:02.205412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.205694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.206032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.207440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=66a7eff73b85406a8874547263816a68) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:02.207758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:02.209174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa68d13-704a-4096-8040-238124e4b37a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:02.222682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 377/773] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:01 2022] GET /identity/v3/role_assignments?user.id=66a7eff73b85406a8874547263816a68&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:02.222682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.222682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.222682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.257300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:02.258528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:02.327431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d838b5-cfd5-4f2b-805f-b8bb9037ac8f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:02.368810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:02.369535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.369827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.370109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.370852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:02.372311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:02.379412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 397/774] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:02 2022] GET /identity/v3/services => generated 772 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:02.391997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 378/775] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:02.427462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:02.459680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:02.563541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935d5e-b990-4a91-8428-c839a55e1b7f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:02.614768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:02.615955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.616454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.616939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] PATH_INFO: `/v3/services/network` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.617812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:02.619475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:02.638078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:02.639154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:02.640004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:02.643044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 398/776] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:02 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:02.648913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:02.761957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:02.767636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a4de-de6b-4bfa-9013-c01537ee54aa None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:02.802933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:02.803608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:02.803895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:02.804178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:02.804900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] RBAC: Adding query filter params (name=network) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:02.811064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:02.812466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:02.818025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 379/777] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:02.823518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:02.920700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:02.942603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caac32f-cdfe-42f7-9845-09b788dcdc3c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.000408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.001473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.002050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.002533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.003514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] RBAC: Adding query filter params (type=network) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:03.004026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:03.005653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:03.015217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 399/778] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:03.021302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:03.155244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83a6dd-dd85-411e-9b23-df71c962ce65 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.200224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.200971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.201221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.201537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.202344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] RBAC: Adding query filter params (interface=public, service_id=cc061d7b28e14e828fb5abaa50568417, region_id=RegionOne) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:03.202681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:03.204046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3836d-48c7-47cb-beec-498a25fd40b4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:03.213376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 380/779] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/endpoints?service_id=cc061d7b28e14e828fb5abaa50568417&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:03.218470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:03.242336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572a4e9-33bf-48f0-ae4b-07f678483961 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.242693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572a4e9-33bf-48f0-ae4b-07f678483961 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.242953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572a4e9-33bf-48f0-ae4b-07f678483961 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.244598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 381/780] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:03.249574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.249862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.250146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.267344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:03.269002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:03.292309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:03.347240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 382/781] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:03.364850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:03.455136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb606d-68cd-4477-8224-55a7498773ed None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.465951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cecf8-ed86-4d6f-95bd-f57891208902 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.501834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.503429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.503545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.503777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.517080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:03.524944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:03.525883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:03.526771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 383/782] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:03.534992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.535673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.535964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.536249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.536946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:03.538335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:03.540157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:03.547083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 400/783] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/services => generated 772 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:03.566349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:03.654148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ca314-3980-4dbc-a26c-cad9088f9384 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.691880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.692564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.692850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.693129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.693894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:03.694207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:03.695667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:03.709793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 384/784] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:03.711236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:03.728187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:03.836859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfc10-6910-4e20-9eb7-a89b131698ec None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:03.883510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:03.884311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:03.884597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:03.884872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:03.895655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:03.897129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:03.916314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:03.925557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:03.926504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:03.926504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:03.926504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:03.926504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:03.926504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 401/785] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:03.926821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.010363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:04.032606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597ee26-958f-44df-9e4e-1da159601a81 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:04.069366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:04.070037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.070325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.070604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.071370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:04.071684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:04.073053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:04.093015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 385/786] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:04.099057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.200522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:04.243889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c643d-d52c-42e4-bee7-83a8ebbd3530 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:04.286871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:04.288062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.288578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.289062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.297069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:04.298794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-3d18d321-6c83-403a-89f0-ed602c15c548 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:04.328210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:04.329213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:04.330185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:04.334671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 402/787] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:04.343792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.377269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542082cd-01c6-4fa7-9557-48c78f5a2ef1 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.377619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542082cd-01c6-4fa7-9557-48c78f5a2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.378403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542082cd-01c6-4fa7-9557-48c78f5a2ef1 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.380153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 403/788] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:04.385090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.385387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.385671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.415952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:04.417260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:04.454801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd974-90f6-4bd8-893d-0d8e1199af2a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:04.491198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:04.491874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.492165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.492447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.493296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:04.493611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:04.495079 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:04.515004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:04.517838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 386/789] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:04.520098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.556716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 404/790] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:04.580985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.684342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:04.690212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cbe88-c9f5-4eb6-99e3-5ae4fd52302c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:04.730519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:04.731369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.731659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.731943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.757279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:04.759158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:04.778593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420ad19-9712-4296-8cf3-afd3023a76de None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:04.806896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:04.807850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:04.808726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:04.808726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:04.808726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:04.808726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:04.813249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 387/791] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:04.818811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.820755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:04.821875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:04.822167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:04.822453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:04.849118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:04.867634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:04.868591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:04.869416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:04.871044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 405/792] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:04.876780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:04.929426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.001372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8f335-62ba-4a22-a21f-110bda5227ee None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.033416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cea33-49d0-42e6-98df-4095b5b207a7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.039329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.040250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.040769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.041251 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.042338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:05.042895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.044654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4795bf-d21c-49c1-8ab3-174335d4cb3f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.051332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.061296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 388/793] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:05.063099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0e76d-0bab-40f8-8000-322d4b8932f9 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.084659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0e76d-0bab-40f8-8000-322d4b8932f9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.084659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0e76d-0bab-40f8-8000-322d4b8932f9 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.084659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 389/794] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:05.084659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0e76d-0bab-40f8-8000-322d4b8932f9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.084962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.085907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.086381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.087132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.088209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:05.088704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.090395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33ec4-9f0f-49d1-a1be-622d3ee3165f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.107194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 406/795] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:05.108816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.109291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.109847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.126909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:05.129021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:05.196244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0e76d-0bab-40f8-8000-322d4b8932f9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.211051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.211241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 407/796] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:05.212112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.249612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.251285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.251632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.251857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.259544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.261002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:05.276735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:05.277844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:05.278838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 390/797] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:05.279331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.347056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aef98-677c-4627-9467-c4694501bf8b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.354580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.380926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.381728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.382016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.382304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.392469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.394489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.395115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bbec5-dedf-4625-8fec-13015ae714ba None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-2304100e-6c5b-4016-a55d-08ab51783bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:05.412385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:05.413298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:05.414199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:05.414199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:05.414199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:05.414199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:05.418365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 408/798] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:05.419703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b395-9a29-4c0b-abfc-4ceca29f5c0a None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.419997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b395-9a29-4c0b-abfc-4ceca29f5c0a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.420278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b395-9a29-4c0b-abfc-4ceca29f5c0a None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.421584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 409/799] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:05.423422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b395-9a29-4c0b-abfc-4ceca29f5c0a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.429245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.430044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.430332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.430612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.441594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:05.449436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:05.450442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:05.451337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 391/800] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:05.452731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.484131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.536891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b395-9a29-4c0b-abfc-4ceca29f5c0a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.559196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266875-c7e8-4da7-85c2-d8ad49d20f86 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.571330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.571994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.572285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.572578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.573358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:05.573675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.575148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d19aac-05f9-4aff-8456-cee90afbd57a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.587338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 410/801] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:05.588561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.588855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.589137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.601407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.602346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.602850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.603314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.604236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:05.604835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.605642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:05.606586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.606869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:05.621297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 392/802] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:05.622413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.626190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.683340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 411/803] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:05.684008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.737994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.749746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33e9b4-be48-41a0-b85f-7d6a06d785d5 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.785637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.786655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.786996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.787229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.787778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293ee08-6919-4b12-b239-a88d0496c6aa None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.788226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:05.788307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.789629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.802677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 393/804] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:05.802677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.831500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.832723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.833274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.833562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/a0f1db595d804131bf47d7ae396d5889/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.853262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=a0f1db595d804131bf47d7ae396d5889, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.854809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.903070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64ddb8-9008-47b6-88c1-31a6fa9df8d4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:05.926163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:05.940848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 412/805] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:05 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/a0f1db595d804131bf47d7ae396d5889/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:05.941885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.944697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:05.945620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:05.945889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:05.946367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:05.954922 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:05.961136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:05.975885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:05.976975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 394/806] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:05.978608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:05.994565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.107583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c956-15e9-4625-8bb5-f7b5f9164f68 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.107943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a05e4-2313-470c-9029-a058addd9b2d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.151879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.152700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.152982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.153254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.154139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.155377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.161420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.162583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.164004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.164486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:06.169642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:06.170584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:06.171857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 413/807] 10.222.0.74 () {56 vars in 1129 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:06.172713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.182855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.184328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:06.206275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:06.207387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:06.208501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:06.208501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:06.208501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:06.208501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:06.209441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 395/808] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:06.210532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.248430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.291465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4d3-c8a7-4276-800d-973887e9dba3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.298684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.330669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.332120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.332489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.332849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.333688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:06.334371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.335867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.351571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2eb4-b7f3-452a-a72b-fe99aa576a54 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.358770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 414/809] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:06.360315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.387108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.387775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.388060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.388342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.389048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:06.389359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.390770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568637ab-a1b1-49f0-9569-87ec9948b9de None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.396111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 396/810] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:06.397333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24aab3-ea92-4ce4-9ca6-7edcf7cf2d17 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.397623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24aab3-ea92-4ce4-9ca6-7edcf7cf2d17 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.397900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24aab3-ea92-4ce4-9ca6-7edcf7cf2d17 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.402768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 397/811] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:06.402768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24aab3-ea92-4ce4-9ca6-7edcf7cf2d17 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.478412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01510df7-6a0e-43ea-8ddf-5b048a6d8d35 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.499345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.512674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24aab3-ea92-4ce4-9ca6-7edcf7cf2d17 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.513932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.514834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.515277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.515560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.516284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:06.516600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.518122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.530710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 415/812] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:06.531830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.545104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.553996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.557118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.557118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.557118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.626810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.628964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.637724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614974-99be-4bd6-8bc0-8a496a30af86 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.672017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.672685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.672968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.673248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.673948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:06.674259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.676301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f2950-6e62-49b4-9769-065789c243ba None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.681314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 416/813] 10.222.0.74 () {56 vars in 1124 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:06.682523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.682839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.683121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.699213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:06.700398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:06.742972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.752520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 398/814] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:06.752520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.776209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 417/815] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:06.777099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:06.790198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:06.868868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57e806-7f0b-44de-944d-6d229cee8490 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.878842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45462c-e074-4b23-b765-3a84d057eaf1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:06.903230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.904022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.904307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.904582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] PATH_INFO: `/v3/users/swift` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.912136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.913503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:06.913503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:06.913689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.913764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:06.913959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:06.914718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:06.916075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:06.929110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:06.930347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 399/816] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:06.936342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.020246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.024433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 418/817] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:07.025410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.047206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757a919-c194-46cf-9215-5e4208ef707d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.075109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.105606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.106894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.107321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.107829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.120400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.122489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:07.147990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:07.148903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 400/818] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:07.149945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.174304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0a4bb-6e77-4ee9-a313-a85219a0a87d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.213242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.214335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.214867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.215410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] PATH_INFO: `/v3/services/network` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.216247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.217834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:07.232333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:07.233559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:07.234521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 419/819] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:07.235417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.285733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e08e7-81d2-4ae4-bbb0-b47787cd6a4e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.334923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.336245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.337115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.337754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.338979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.339728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.342115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.344667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.357871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.371684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 401/820] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:07.371684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.371917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e9c4a-85a8-4101-9a55-4c9a1e961883 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.414148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.415178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.415469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.415752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.416478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.416791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.418796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.437937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 420/821] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:07.438836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.473517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf9483-e705-4b57-8c51-35f23ecca6ff None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.522906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.523577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.523862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.524138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.524856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] RBAC: Adding query filter params (name=network) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.525164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.526529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.533386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 402/822] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:07.534246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.558288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c48039-4ec0-4027-a323-7488fdb27d0a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.594959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.596010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.596356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.596764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.607684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.611075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.613166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:07.638278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:07.639485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:07.640387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:07.640387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:07.640387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:07.640387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:07.640387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 421/823] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:07.641026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.658471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.663089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0cfce-bbbb-4873-a304-a87ddf63745c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.699919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.700735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.701018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.701296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.710590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.712197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:07.727320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:07.728378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:07.729439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:07.730857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 403/824] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:07.731850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.776055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48466d55-2ea8-4a51-8007-79d49a0c4cf0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.816029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.816951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.817376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.817792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.818670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] RBAC: Adding query filter params (type=network) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.819690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.821238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.829086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 422/825] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:07.830339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.847501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d6114-3253-4f6d-b5bd-852b95790660 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.866195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.891888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.892944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.893406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.894090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.895019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.895469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:07.896970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:07.914227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:07.918671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 404/826] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:07.919720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:07.956390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e078db-1f67-4213-9545-6388dfec13b4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:07.995872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:07.996698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:07.997086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:07.997466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:07.998350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:07.998835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.000379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.013697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 423/827] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:08.014786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.050809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112985-d0fd-460a-9a5b-f27c66e822fd None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.099909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.100937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.101366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.101808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.103295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.105135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.123845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a637-2ac6-4da6-bb5f-51fefeabe744 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.154317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:08.160691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.161594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.161979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.162352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.169349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.170974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:08.185199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:08.186233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:08.187335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:08.193997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 424/828] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:08.193997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.204064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 405/829] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:08.205620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.254697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:08.344175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a34f-7fc0-4247-81c5-34c29d18f3d2 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.367256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9c2e5-3ff7-4472-ba62-de2a61edf117 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.386052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.387274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.388239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.388704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.390057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:08.390839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.392915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061eee1-197a-49cf-8b84-3e77125914ea None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.401819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 425/830] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:08.403404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922908a0-4ecc-4506-90ae-8a5acd459da8 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.403840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922908a0-4ecc-4506-90ae-8a5acd459da8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.404263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922908a0-4ecc-4506-90ae-8a5acd459da8 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.405866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 426/831] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:08.412235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.412594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.412796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.414884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.415677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.415959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.416237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.417160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:08.417478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.419097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.434243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:08.435554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 406/832] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:08.436267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:08.442846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.502405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:08.531241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 427/833] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:08.552890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.577434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:08.595396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00eacb-741d-4350-b48c-312fdf5eb61c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.637194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.638087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.638371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.638681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/66a7eff73b85406a8874547263816a68/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.640646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=66a7eff73b85406a8874547263816a68, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.642085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.679755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee0821-d633-447f-a665-9ba0ac08951c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.714724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 407/834] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:08 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/66a7eff73b85406a8874547263816a68/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:36:08.720442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.720442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.720686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.720931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.735436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:08.743590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:08.744647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:08.745495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 428/835] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:08.750379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.876840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7668ad-b3c2-4170-bdae-b74b21fa143f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:08.916510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:08.917808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:08.918239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:08.918899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:08.919835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:08.920343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:08.921869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:08.931560 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 408/836] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:08.937617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:08.994094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.051810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3d744-94b1-4924-a1d1-f59f56c8d0d3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.087801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.088595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.088878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.089159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.100195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.101672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:09.119457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:09.120559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:09.121625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:09.121625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:09.121625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:09.121625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:09.121625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 429/837] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:09.126558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.140686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.235215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b7dd-b4c0-4d97-8821-ba47f364f2db None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.272816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.273484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.273764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.274402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.274802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:09.275118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.276486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71212e-5ecf-4f18-b1e0-51bf9f54c49c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.289274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 409/838] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:09.294761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.395538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.401162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a563-c3d9-4814-b7c7-0271543af369 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.411395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dda3-5f47-4851-83c0-36aed0ed9c3d None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.411685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dda3-5f47-4851-83c0-36aed0ed9c3d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.411966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dda3-5f47-4851-83c0-36aed0ed9c3d None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.413511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 410/839] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:09.418621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.418943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.419224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.438301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.438755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:09.439173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.439449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.439737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.439952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:09.447829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.449511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:09.465063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:09.465997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:09.466969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 430/840] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:09.472213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.515252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.522722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 411/841] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:09.542628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.606001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd074d-f48e-4a51-84d6-2402e348c0e5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.654145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.654921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.655214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.655498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.656773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:09.657242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.658704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.659701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85084ff-a0f0-4aec-b1b4-4438036bfd2a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.681218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 431/842] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:09.686367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.702070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.705135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.706401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.706896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.707320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.708137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.710698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:09.726844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:09.728135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:09.729163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:09.738360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 412/843] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:09.740779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.802078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc90d7a-28a0-4f2d-b8e2-24f4749b8e3a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.822681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:09.840091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.840743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.841344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.841344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.875175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599038-d4ad-4820-895f-b13371baab27 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:09.922452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.923869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:09.924006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.925377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:09.925824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:09.926511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:09.928308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:09.928773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:09.930293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:09.941562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 413/844] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:09.943910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:09.945284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:10.055321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3784c02-a6ad-442f-9be6-89343f57ad99 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.068015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 432/845] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:36:10.072969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:10.087976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:10.096200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.098749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.098749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.098749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.098749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:10.098749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.100343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.110665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 414/846] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:10.198453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63093b4a-2a08-4f8f-9979-e0968a2a1d91 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.214618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:10.236678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.238217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.238600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.239010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.251196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.253394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:10.273015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:10.274159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:10.275273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:10.275273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:10.275273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:10.275273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:10.276886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 433/847] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:10.282287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:10.403746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bccd8-ec9d-426a-a47b-5ef6ad2e1cea None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.439955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.441428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.441856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.442357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.443296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:10.443751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.445273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.458872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 415/848] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:10.466736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:10.564170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6492-1ab8-4cdc-a4ce-ec6b49fd6379 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.604960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.605762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.606043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.606316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.613319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.614798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:10.629175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:10.630063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:10.631399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 434/849] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:10.636059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:10.697731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:10.749127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d04fc-e2e5-4e80-9f14-34cd7914dd37 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.784798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.785457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.785737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.786014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.786943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:10.787313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.788799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975867e-78a5-44d2-817d-88bf9ca6277f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.801202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 416/850] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:10.807066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:10.835697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:10.883382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0e87-f244-46d5-b039-b8d58b57b6f3 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.883833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0e87-f244-46d5-b039-b8d58b57b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.884042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0e87-f244-46d5-b039-b8d58b57b6f3 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.885687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 417/851] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:10.891532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb36fc5-d5ba-456f-a3a7-16409de0a9b5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.891853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb36fc5-d5ba-456f-a3a7-16409de0a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.892097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb36fc5-d5ba-456f-a3a7-16409de0a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.915106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bdb0-734a-4e8b-8d71-4da58da37514 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:10.918258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-bbb36fc5-d5ba-456f-a3a7-16409de0a9b5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:10.919484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bbb36fc5-d5ba-456f-a3a7-16409de0a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:10.951959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:10.952606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:10.952889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:10.953166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:10.954188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=a0f1db595d804131bf47d7ae396d5889) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:10.954508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:10.956527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:10.963539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 435/852] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:10 2022] GET /identity/v3/role_assignments?user.id=a0f1db595d804131bf47d7ae396d5889&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:11.008829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 418/853] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:11.031063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.077923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d52f-fcc0-40ad-beb7-a8002effaa96 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.078357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d52f-fcc0-40ad-beb7-a8002effaa96 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.078767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d52f-fcc0-40ad-beb7-a8002effaa96 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.081812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 419/854] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:11.090155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.090695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.091116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.109643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:11.111188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:11.145720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.155625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132339b0-4b5f-4b7a-a3bb-74e0b4e3bbd0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.170211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.194481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.195589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.196039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.196465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.197374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.198793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.211384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 420/855] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:11.212579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:11.213491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:11.214307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:11.217630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 436/856] 10.222.0.74 () {56 vars in 1132 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:11.224341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.237252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.343803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ac9db-af53-40aa-baa3-d9e9ed0a4542 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.381006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.381962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.382385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.382842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.383677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a54dde-6de4-4c41-bd07-c13326ea6ceb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.383856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:11.384345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.385843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.391584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 421/857] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:11.397867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.426909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.428174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.428779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.429133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.435848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.444620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:11.454829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:11.456401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:11.457288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 437/858] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:11.463364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.484693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.517285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f231d-40d2-457d-b399-53e9a27a4a9f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.555473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.556468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.556959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.557430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.558364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:11.558894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.560446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.570951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 422/859] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:11.591694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241847c-83d7-432b-90db-ba6fb356fc2c None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.631564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.632258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.632550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.632832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.633598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:11.633906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.635360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.645844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 438/860] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:11.651553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.761203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.764411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f2b66-64c1-4f4a-b05c-64b3297104ca None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.800892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.801686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.801983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.802249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.812913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.814366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:11.832926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:11.833836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:11.835037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:11.835037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:11.835037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:11.835037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:11.835037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 423/861] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:11.840298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.926151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.950714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7afda-d9f9-4ce0-b1cf-b7ac21c1930d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.986737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.988092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.988737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.988737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.990694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:11.990694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.991182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.004270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 439/862] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:12.009864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.136977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36462e-d95c-4205-8fef-c67eed234de4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.178657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.179391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.179676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.179953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.187428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.188852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:12.203642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:12.204606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:12.205527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 424/863] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:12.216259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.363367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa59bfc-8b1a-407d-a7a0-e55da669f335 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.407832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.408523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.408869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.410677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.410677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:12.410863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.412266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf83528-624c-413f-b5b4-bc0e9b0bdbae None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.423486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.451534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 440/864] 10.222.0.74 () {56 vars in 1135 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:12.460376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.485668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.508744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcc9ca-85ed-4bf3-a47c-d0796c190d2c None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.509060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcc9ca-85ed-4bf3-a47c-d0796c190d2c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.509304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcc9ca-85ed-4bf3-a47c-d0796c190d2c None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.510949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 441/865] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:12.520393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.520671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.522676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.547573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:12.548775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:12.639893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 442/866] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:12.668659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.675538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30fb1-24f4-4f50-94c0-bdc907df0176 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.712950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.714291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.714291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.714291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.785465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1149ca8-435b-486a-9875-e10fc38a8891 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.786236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.787843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.802839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.813367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.825757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.825757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.826302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.826764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.827629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.829252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.884425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 443/867] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:12 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:36:12.908010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 425/868] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:12.915774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.032833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d294953-4dfc-4c9d-a2ba-9e4c99fb5a87 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.069510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.070321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.070601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.070902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.082979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.084789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:13.103306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:13.104256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:13.105152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:13.105152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:13.105152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:13.105152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:13.105152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 444/869] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:13.110590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.219255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7ae2b-ff2f-47ec-904d-b0b8cf0b2728 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.233161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:13.255674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.258785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.258785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.258785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.258785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:13.258785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.259352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f723eb-7233-4831-a844-1078e2644dee None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.271657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 426/870] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:13.277419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.310409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:13.388092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4a40-aff9-4ea9-8ad4-c67fd5ee9835 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.388430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4a40-aff9-4ea9-8ad4-c67fd5ee9835 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.388648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4a40-aff9-4ea9-8ad4-c67fd5ee9835 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.390156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 427/871] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:13.395729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.396012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.396290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.398823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60f6-ef67-47da-a85a-6417e43d35c7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.416286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:13.417466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:13.441399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.442311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.442596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.443036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.451284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.452978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:13.467732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:13.468660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:13.469535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:13.472548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 445/872] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:13.478148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.497243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 428/873] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:13.520401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.605893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:13.615921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e375d9-b4e8-4384-ab3b-9742d5c1a8e9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.630089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4edf0-40ac-4b7c-98b8-b7c03dfd68ba None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.638422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:13.656346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.657011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.657393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.657569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.658341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:13.658666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.660629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.667803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.668906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.669404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.669877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.674467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 446/874] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:13.683659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.689608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:13.698574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:13.699527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:13.700353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 429/875] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:13.707585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.810745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe07ffd-c75b-4b0d-9f6b-bd58463b2552 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.847457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.848461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.848944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.849443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.850525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=66a7eff73b85406a8874547263816a68) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:13.851071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.852693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.854966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135be542-8e2b-4e4a-97f9-66404021af99 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.867057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 447/876] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/role_assignments?user.id=66a7eff73b85406a8874547263816a68&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:13.894728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.895412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.895694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.895974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.896733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:13.897044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.898452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852c9b4-9f8b-440d-83c7-bb7a55736a2d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.908577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 430/877] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:13.914743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.936365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:13.950269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de7eb0-7573-42b3-bb43-e5535dfbe7e1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.950736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de7eb0-7573-42b3-bb43-e5535dfbe7e1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.951024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de7eb0-7573-42b3-bb43-e5535dfbe7e1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.953353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 431/878] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:13.961265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.961556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.961835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.980817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:13.982050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:14.002159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:14.049688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7ba4-fc31-47f9-88a8-421f81bac620 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.066055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 432/879] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:14.090088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:14.093193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.094913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.095203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.095481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.109139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.112319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:14.129601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:14.130468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:14.131666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:14.131666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:14.131666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:14.131666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:14.131666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 448/880] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:14.137118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:14.225407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:14.231422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0ab3-9919-4886-beb1-89fa593100de None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.270374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495fd6-5d0a-402f-8b46-570e5d7b3301 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.273641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.274548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.275044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.275460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.276873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.279008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:14.308797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.309463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.309751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.310029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.310787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:14.311152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.312476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:14.325042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 449/881] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:14.330691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:14.381674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 433/882] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:14 2022] POST /identity/v3/services => generated 269 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:14.452630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d84f-541d-4d94-b0f9-13ea2a00920b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.484426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:14.499184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.500021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.500306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.500586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.509466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.511065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:14.528279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:14.529329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:14.530262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 450/883] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:36:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:14.537488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:14.555166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:14.646865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f86a1-01a3-4783-b2b7-a77a5ae92762 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.682601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.683353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.683640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.683917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.684763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:14.685075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.686506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:14.705080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 434/884] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:36:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:14.711512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:14.819883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce5623-963e-4722-b36f-94a6e5d254f5 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:14.856156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:14.856806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:14.857089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:14.857367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:14.920796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:14.931978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:14.933502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.034881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 451/885] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:15.040593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.123636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.158495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f374-a4e8-4497-a165-c7aaa92419cb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.197631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.198345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.198674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.198939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.210330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.211813 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-cefef454-b6a8-4fb7-8fb0-858febbdcd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:15.229462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:15.230367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:15.231295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:15.231295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:15.231295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:15.231295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:15.231295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 435/886] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:15.237643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.306248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601ed9-9cff-4d76-a94c-90bcb0a35105 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.306576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601ed9-9cff-4d76-a94c-90bcb0a35105 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.306851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc601ed9-9cff-4d76-a94c-90bcb0a35105 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.308468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 436/887] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:15.318886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.319162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.319446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.332798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.340023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:15.341512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:15.345264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3605cc6-d5bb-4f53-9e20-0244b14c613b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.381315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.381979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.382263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.382540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.383306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:15.383614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.385046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.396641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 452/888] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:15.403524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.420000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 437/889] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:15.439088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.505150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.524216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54a2c-a0e5-4c7e-860e-b9113ebf0f35 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.561149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83f296-ee90-4b8c-9726-b8711f48d2cc None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.561587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.562683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.563144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.563691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.571574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.573380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:15.589458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:15.590408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:15.591661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:15.594211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 453/890] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:15.600694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.605174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.611205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.612601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.612887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.613161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.613823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.615236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:15.628643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:15.629632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:15.630456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:15.635167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 438/891] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:15.636130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.717320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6add-efd7-44b0-9da3-7e324a6a280f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.741247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186a498-7741-4366-bd57-c4c8b0ef4847 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.756406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.757342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.757919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.758374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.759164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.760554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:15.761049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.762838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.776542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 454/892] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:15.777451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.778354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.778839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.779319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.780207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:15.780718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.782207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.784540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.795123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 439/893] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:15.806524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:15.867580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:15.915970 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20872614-6ef7-40d2-b66e-45399197a3fc None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.944501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f9769-dd66-4aa1-acd7-8bb1ca5fcd79 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:15.960464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.965731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.965731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.965731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.965731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] RBAC: Adding query filter params (role.id=d4196859f6d445d990d30580d822baec, scope.project.id=c07159d21300478e88a07006614fdded, user.id=a0f1db595d804131bf47d7ae396d5889) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:15.965731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.966117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:15.978677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 455/894] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/role_assignments?user.id=a0f1db595d804131bf47d7ae396d5889&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d4196859f6d445d990d30580d822baec => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:15.991364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:15.992054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:15.992342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:15.992623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:15.993531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:15.993846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:15.995353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.001832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 440/895] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:16.007959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.124339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0bb06-76ef-4e48-b123-a97e0ab7b569 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.157256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:16.167436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] RBAC: Adding query filter params (interface=public, service_id=6460b336322e47cbaefea1dcbf91316f, region_id=RegionOne) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:16.192441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1f609-9f9f-49c6-91e9-9e886d4df079 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.192988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 456/896] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/endpoints?service_id=6460b336322e47cbaefea1dcbf91316f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:16.193064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.239165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:16.251442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5027a-2dc1-4b14-a89f-f5eef9451572 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.252292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5027a-2dc1-4b14-a89f-f5eef9451572 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.252672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5027a-2dc1-4b14-a89f-f5eef9451572 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.254164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 457/897] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:16.266868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.266868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.266868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.282919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:16.284735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:16.298492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62924e54-75eb-4778-8067-bb7c184114e3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.343931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:16.344996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.345507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.346003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.347495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:16.349179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.362665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 441/898] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:36:16.362887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 458/899] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:16.384429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.494756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc710a9-c503-4522-9668-390fd7e50776 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.507141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:16.535811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:16.536578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.536864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.537149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.537832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:16.539324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-db1dba71-fe85-46a1-93ba-d5666feff78b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:16.552628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:16.553782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:16.555085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 442/900] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:16.560299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.672441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09e5c-ca99-4668-b058-e63ac019fa3c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.696963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:16.709847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:16.712967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.712967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.712967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.712967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:16.712967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:16.713482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf0e6e-676e-4b1f-a622-ba6540c8932b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.718696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f26194-fc2a-4fb5-81d4-e06f8b23af5c None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.718965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f26194-fc2a-4fb5-81d4-e06f8b23af5c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.719237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f26194-fc2a-4fb5-81d4-e06f8b23af5c None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.720846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 443/901] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:16.721322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 459/902] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:16.726264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f26194-fc2a-4fb5-81d4-e06f8b23af5c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.727183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.727725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.728224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.745784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:16.747334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:16.830843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 460/903] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:16.848587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:16.851415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f26194-fc2a-4fb5-81d4-e06f8b23af5c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.888713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:16.889342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:16.889618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:16.889909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:16.890598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:16.890937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:16.892301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:16.902445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 444/904] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:16.971406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb860db7-a90e-49f1-8c99-f9106bf19bb1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:16.995180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:17.016525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:17.017357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:17.017645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:17.017927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:17.018886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:17.020302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:17.033998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:17.035065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:17.036100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 461/905] 10.222.0.74 () {56 vars in 1132 bytes} [Wed May 25 14:36:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:17.043756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:17.088785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:17.161002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87d66-4ebd-4b91-9431-ed2786019185 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:17.198480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:17.199283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:17.199488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:17.199768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:17.200487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:17.200799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:17.202212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:17.208224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 445/906] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:17.214341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:17.340194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b8a4a-cb15-4610-a503-f571395b6371 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:17.376086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:17.376756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:17.377042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:17.377320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:17.378096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:17.378408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:17.379855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:17.385061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 462/907] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:17.392897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:17.500080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd999b07-91ee-4efa-81c0-99256c2dec3c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:17.502057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:17.535965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:17.536671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:17.536955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:17.537232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:17.538096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] RBAC: Adding query filter params (interface=public, service_id=5c983658d78f442b9e84dd6a2c559859, region_id=RegionOne) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:17.538414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:17.539817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c48c4b-cb54-4f25-ba9c-aa1946a5d08d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:17.549390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 446/908] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:17 2022] GET /identity/v3/endpoints?service_id=5c983658d78f442b9e84dd6a2c559859&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:17.550764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:17.602331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:17.659668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092e09d-c836-4b0b-ae39-891a3b5490a7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:17.694472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:17.695515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:17.696043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:17.696629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:17.697517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:17.699143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa94fc-6a9b-4890-ac46-ce78ab583b76 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:17.706249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 463/909] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:17 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:36:18.394142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7434e-7150-4032-ae55-a911ff806647 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.394698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7434e-7150-4032-ae55-a911ff806647 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.394792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7434e-7150-4032-ae55-a911ff806647 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.396364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 447/910] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:18.401814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff068-e96e-4cbd-944b-d746ca78fc66 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.402238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff068-e96e-4cbd-944b-d746ca78fc66 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.402436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff068-e96e-4cbd-944b-d746ca78fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.419458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a7bff068-e96e-4cbd-944b-d746ca78fc66 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:18.420651 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a7bff068-e96e-4cbd-944b-d746ca78fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:18.493664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 464/911] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:18.516302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7434e-7150-4032-ae55-a911ff806647 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:18.624232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7434e-7150-4032-ae55-a911ff806647 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:18.660718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:18.661533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.661822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.662100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.672768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:18.681542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:18.682503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:18.683373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 448/912] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:18.684265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a80f7-7252-4a36-a2a1-3d908b6c677d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.684713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a80f7-7252-4a36-a2a1-3d908b6c677d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.685125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a80f7-7252-4a36-a2a1-3d908b6c677d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.686832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 465/913] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:18.691875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:18.692462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.692964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.693437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.710606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:18.712132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:18.720838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:18.748891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:18.806707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33122fa-d89c-4f63-a7b2-7ee2f07d714a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:18.806959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 466/914] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:18.826528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:18.845723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:18.847018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.847306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.847585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.848353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:18.848668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:18.850098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:18.859637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 449/915] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:18.865715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:18.952607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8b635-a6d2-4279-a181-69cf54e2f6a6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:18.972220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:18.985901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cdcf-2cb6-4ecd-abd2-390db334f30d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:18.995317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:18.996405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:18.996901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:18.997382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:18.998438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.001207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:19.014224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:19.015250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:36:19.016069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:19.020590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 467/916] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:19.026782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.026782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.026782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.026782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.027138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.038599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.039994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:19.059689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:19.060632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:19.061526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:19.061526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:19.061526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:19.061526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:19.065931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 450/917] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:19.066924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.068191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.139160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccbc5-0bba-48d3-8838-6d86ffd97877 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.177146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2055d-71be-423a-beab-ccb581a2cc2e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.178033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.178855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.179028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.179307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.180671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:19.180671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.181648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17e73a-711b-493b-af84-ee8238aac6fa None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.191118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 468/918] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:19.192183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d15c-f60d-4786-9418-434027005cfd None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.192483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d15c-f60d-4786-9418-434027005cfd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.192763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d15c-f60d-4786-9418-434027005cfd None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.193987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 469/919] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:19.194696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25d15c-f60d-4786-9418-434027005cfd None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.218563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.219334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.219691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.220034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.220820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:19.221740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.223186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a964c8-bdef-4da1-bc8a-c466e475fdbf None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.231568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.240491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 451/920] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:19.240491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.240491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.240491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.256447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:19.257694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:19.302342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.311123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25d15c-f60d-4786-9418-434027005cfd None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.342672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 452/921] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:19.342672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.352548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.353618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.353910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.354188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.354918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:19.355590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.357278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.363409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 470/922] 10.222.0.74 () {56 vars in 1114 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:19.368371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.458798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c5b59-5b12-43c3-a705-4d70538bb771 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.475965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80776a5-e5d6-470d-b893-a37464fe66bd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.490983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.497748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.498735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.499022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.499382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.506995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.514819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.514923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.514923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.514923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.514923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.515231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.515231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:19.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:19.526015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 453/923] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:19.527021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.582709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 471/924] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:19 2022] POST /identity/v3/services => generated 241 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:19.582709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.611392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.648885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0e4b-d4bc-44af-896b-50410e05a829 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.700142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.701161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.701625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.702114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.703126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.704693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.727004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee74c-1f14-4b20-b187-aa32aebf86ca None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.765722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.766406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.766715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.766994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.767828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:19.768708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.770149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea858-5253-42f7-af4c-58c69bfc1edf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.784435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 454/925] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:19.791819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 472/926] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:19.798112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:19.856898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:19.926803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a0a1-e6e4-402e-8e65-3557e2ebe3cb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:19.971304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:19.972710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.973192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.973679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.982919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576f71-44e4-444f-a07c-74de13ae61f4 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.983202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576f71-44e4-444f-a07c-74de13ae61f4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.983418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db576f71-44e4-444f-a07c-74de13ae61f4 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:19.986521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 473/927] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:19.987193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:19.988708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:19.990878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:19.991384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:19.991882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:20.007518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:20.008474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:20.009960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:20.009960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:20.009960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:20.009960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:20.009960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 455/928] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:20.014037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.021140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:20.022282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:20.096381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 474/929] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:20.116227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.119396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.125861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01baa7-a1d2-430e-86a6-1c6ed50e26da None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.137813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.164877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.167580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.168750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.169142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.170056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:20.170476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.172018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.190684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 456/930] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:20.192508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.248824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55627064-26d8-4cc7-b699-17e886bb1df1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.286915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.288267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.288555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.288827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.289765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.291797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.301104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0a8c0-3865-43ab-ad80-17a3d5cc0c8a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:20.308574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:20.309491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:36:20.310337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:20.316729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 475/931] 10.222.0.74 () {56 vars in 1132 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:20.318423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.349138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.350044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.350335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.350618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.358654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.360123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:20.375874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:20.377938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:20.379083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 457/932] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:20.384754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.399343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.402759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.434069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d76d4b-7e55-403f-9103-a1dab16d04f8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.469939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.470767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.471695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.472048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.472755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:20.473062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.474439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.484464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 476/933] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:20.486073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.500630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdb05-adf4-4f84-90ea-e7d7246be1d0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.541851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.542674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.542944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.543230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.544017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:20.544328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.545819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.564296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 458/934] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:20.565301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.618982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77da3d-71e2-4c05-982e-213896039ea6 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.648254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.666366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.667381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.667844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.668315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.669622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:20.670395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.672454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.675624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:20.684423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 477/935] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:20.685565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:20.688508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeea46-1e21-4d07-b7d3-d2e6325cddd7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.727423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.728366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.728650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.728929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/a0f1db595d804131bf47d7ae396d5889/roles/d4196859f6d445d990d30580d822baec` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.731404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=a0f1db595d804131bf47d7ae396d5889, role_id=d4196859f6d445d990d30580d822baec)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.732837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae00ef-5763-4f55-a605-30b7f49c184a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.798527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd500a-2364-4a68-8284-8e2f9322b1d3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:20.818964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 459/936] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:20 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/a0f1db595d804131bf47d7ae396d5889/roles/d4196859f6d445d990d30580d822baec => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:20.852191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:20.852928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:20.853225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:20.853568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:20.854341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:20.856140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9f600-51cc-41b1-9f0c-51a684d67c1d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:20.951308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 478/937] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:21.931746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec023e4f-607b-4694-95f3-b9e0b62109a1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:21.932315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec023e4f-607b-4694-95f3-b9e0b62109a1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:21.932315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec023e4f-607b-4694-95f3-b9e0b62109a1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:21.933991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 460/938] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:21.939625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dffc9-a3d5-4260-8cfb-7da9c10910ea None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:21.940096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dffc9-a3d5-4260-8cfb-7da9c10910ea None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:21.940267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dffc9-a3d5-4260-8cfb-7da9c10910ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:21.957497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7d5dffc9-a3d5-4260-8cfb-7da9c10910ea None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:21.959066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7d5dffc9-a3d5-4260-8cfb-7da9c10910ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:22.014102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.037684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 479/939] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:22.060696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec023e4f-607b-4694-95f3-b9e0b62109a1 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.104292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ca40c-9cf0-4cf3-8e74-4af2ff4b5541 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.104587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ca40c-9cf0-4cf3-8e74-4af2ff4b5541 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.104874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ca40c-9cf0-4cf3-8e74-4af2ff4b5541 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.106238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 480/940] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:22.111367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.111690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.111981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.128987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:22.130362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:22.189528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec023e4f-607b-4694-95f3-b9e0b62109a1 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.220244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.229547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 481/941] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:22.231155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.235187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.235187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.235187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.235187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.235187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:22.246969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:22.247938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.248777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 461/942] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:22.254320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.256987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.382301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b31d4-48e7-4670-b606-94779740d28b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.388879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ec387-799e-4057-90d0-bcc22aeea4fe None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.404600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.425217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.425980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.426271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.426565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.427296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.428634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.428861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.429916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.430219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING keystone.server.flask.common [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 14:36:22.430687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.431280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.432340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:22.432865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.434763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.442020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 482/943] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:22.447171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.487469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.sql.core [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:36:22.487469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:36:22.487469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [parameters: {'id': 'bae964ad73c44cb88effb3f9eb69fc4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:36:22.487469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bae964ad73c44cb88effb3f9eb69fc4b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bae964ad73c44cb88effb3f9eb69fc4b. May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application conn.query(q) May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:36:22.510439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result.read() May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 14:36:22.511332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application next(self.gen) May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application yield resource May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application next(self.gen) May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise self.value May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application session.commit() May 25 14:36:22.512320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.session.flush() May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application compat.raise_( May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:36:22.513818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:36:22.515111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application conn.query(q) May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result.read() May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:36:22.517196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [parameters: {'id': 'bae964ad73c44cb88effb3f9eb69fc4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.519892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:22.521835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bae964ad73c44cb88effb3f9eb69fc4b. May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 462/944] 10.222.0.74 () {60 vars in 1140 bytes} [Wed May 25 14:36:22 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:22.524029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.571388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2a5a9-23ff-408a-8277-300d21d1cfdf None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.585672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.613427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.614502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.615023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.615523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.616482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:22.617051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.624838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 483/945] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:22.630820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.656128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48192fb-ea39-4c08-bfa1-685d14f087af None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.679688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.703124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.704179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.704474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.704760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.716452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.719244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:22.734539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:22.735633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:22.736550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:22.739192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 463/946] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:22.745144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.759111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c4a8c-f0b2-411d-b67e-6df4b26b6919 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.804911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.805585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.805878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.806165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.807161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] RBAC: Adding query filter params (interface=public, service_id=337c6d612da94d5c99d3ea2ac9c3cdff, region_id=RegionOne) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:22.807490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.809560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.818783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 484/947] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/endpoints?service_id=337c6d612da94d5c99d3ea2ac9c3cdff&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:22.823718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:22.860228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9fd50-3245-4e81-bd6a-872960449be2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.874262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:22.904336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:22.905409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:22.905899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:22.906375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:22.908555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:22.909160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:22.910977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235bc49-accc-4188-bce2-35fc649f528b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:22.938838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 464/948] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:22.970452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c91d-b6e6-4b0d-b53d-c5418ed4e541 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:22.983202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.010442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.011627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.012251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.013007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.013825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.015301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335e969-020d-4da0-b87b-50bc9ccb24de None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.022823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 485/949] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:22 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:36:23.077398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5ac5-eaa5-4eff-8ca3-0d44e9fbdb64 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.077758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5ac5-eaa5-4eff-8ca3-0d44e9fbdb64 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.077960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5ac5-eaa5-4eff-8ca3-0d44e9fbdb64 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.079864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 465/950] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:23.086464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f4d-4efc-4842-b4f4-d307358a1ef3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.086811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f4d-4efc-4842-b4f4-d307358a1ef3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.087079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f4d-4efc-4842-b4f4-d307358a1ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.105980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a5776f4d-4efc-4842-b4f4-d307358a1ef3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:23.107267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a5776f4d-4efc-4842-b4f4-d307358a1ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:23.187972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 486/951] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:23.213189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de5ac5-eaa5-4eff-8ca3-0d44e9fbdb64 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.270537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b47759-af17-4c13-b97f-13b4ce289818 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.270884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b47759-af17-4c13-b97f-13b4ce289818 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.271094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b47759-af17-4c13-b97f-13b4ce289818 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.272700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 487/952] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:23.279308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.279591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.279874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.297189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:23.298760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:23.333030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de5ac5-eaa5-4eff-8ca3-0d44e9fbdb64 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.376259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.377438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.377796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.378150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.385147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 488/953] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:23.391937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:23.400566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:23.401595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:23.402585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:23.405106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 466/954] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:23.410449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.411575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.441247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.450985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.522919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12367b23-f037-4b48-b9c3-b2bc09a8f2b7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.534781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaae683-aaaa-40fa-b126-9e5ada652054 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.573808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.573808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.573808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.574214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.574316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.574399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.574493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.574585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.574585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.574585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:23.574821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.574917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.574917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.584226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 467/955] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:23.589526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:23.589652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:23.590884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:23.592110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:23.592851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 489/956] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:23.598667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.703241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.712171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.714108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954600d6-61fa-4c8c-8193-3d944ee24d31 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.723423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f123-1b72-4d1e-87d9-8e126df16de9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.750672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.751612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.751887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.752324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.763826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.765285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.771155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.772097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.772599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.773210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.774187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:23.775280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.777384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.783567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 490/957] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:23.789306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:23.790365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:23.791365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:23.792289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:23.792289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:23.792289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:23.792289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:23.800362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 468/958] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:23.801472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:23.898302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068b68-4f35-4f38-99c3-c6d084c70b99 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.908695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf4376-81af-41d1-b928-115a456a51b8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:23.935146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.935990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.936459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.936838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.937647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:23.938059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.940309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.947179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:23.948536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:23.949031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:23.949535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:23.950504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:23.950775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 491/959] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:23.951270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:23.952900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:23.967515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 469/960] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:23.974119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:24.000498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:24.087716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff42c0-c1e7-4a8f-84e0-385852b54993 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:24.126588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:24.126588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:24.126911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:24.127197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:24.135129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:24.136612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:24.151843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:24.152901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:24.154378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 492/961] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:36:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:24.162160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:24.166889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:24.274137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16522-d24a-43a2-8250-d6d14ba60795 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:24.309362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:24.310029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:24.310312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:24.310587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:24.311452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:24.313035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:24.314491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:24.334418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 470/962] 10.222.0.74 () {56 vars in 1149 bytes} [Wed May 25 14:36:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:24.339878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:24.424582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:24.444274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c50b9-9353-4683-9a04-392160c1df86 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:24.484085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:24.484994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:24.485425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:24.485857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:24.554187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:24.555969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:24.636383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:24.657598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 493/963] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:24.662518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:24.729645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:24.770974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed99e83-9c0d-45ff-b256-300faf7a5196 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:24.808418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:24.809364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:24.809746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:24.810118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:24.821423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:24.823030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:24.841784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:24.842915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:24.844011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:24.844011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:24.844011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:24.844011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:24.845599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 471/964] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:24.851085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:24.957794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1496d-ce93-44da-92c9-8f622fa76cc6 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:24.992328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:24.993069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:24.993262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:24.993540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:24.994260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:24.994567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:24.996109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.008658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 494/965] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:25.014108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.124819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0cc8e-f7f4-4c63-958f-c0df36286984 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.139815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.161555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.162379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.162723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.162962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.171112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.172608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-0194bfba-89ad-436b-bd01-a1bc0bd356ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:25.189579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:25.191161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.192060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 472/966] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:25.198961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.246967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.307761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec279-148a-4518-96a9-fe05abc1e9e4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.357860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.358527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.358849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.359128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.359510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46936-e11c-49cd-85a8-e7be6ca00e6e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.359802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46936-e11c-49cd-85a8-e7be6ca00e6e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.359919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:25.360095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46936-e11c-49cd-85a8-e7be6ca00e6e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.360200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.362160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.362273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 473/967] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:25.367500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9967e-4a84-4787-a461-1e720bfec61e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.368044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9967e-4a84-4787-a461-1e720bfec61e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.368543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9967e-4a84-4787-a461-1e720bfec61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.375386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 495/968] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:25.383603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.389728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-58f9967e-4a84-4787-a461-1e720bfec61e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:25.391363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-58f9967e-4a84-4787-a461-1e720bfec61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:25.467567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 474/969] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:25.469355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99abd2-83a0-4864-8eff-d48aa994ca96 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.469720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99abd2-83a0-4864-8eff-d48aa994ca96 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.470073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99abd2-83a0-4864-8eff-d48aa994ca96 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.474666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 475/970] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:25.480667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.481060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.481343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.497224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.500370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:25.500876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56749183-e213-42fd-a955-43ffecf6e18f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.501569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:25.537410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.538819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.538819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.538819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.539455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] RBAC: Adding query filter params (role.id=d4196859f6d445d990d30580d822baec, scope.project.id=c07159d21300478e88a07006614fdded, user.id=a0f1db595d804131bf47d7ae396d5889) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:25.539770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.541199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.548084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 496/971] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/role_assignments?user.id=a0f1db595d804131bf47d7ae396d5889&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d4196859f6d445d990d30580d822baec => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:25.548708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.592564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 476/972] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:25.616971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.655014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.677523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da15f-957d-4620-aac0-2228adebe172 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.726238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.727468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.727965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.728448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.729904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.732023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.743111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3345-a02c-4cfd-b384-c32f2bc4815f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:25.748208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:25.749446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:25.750466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:25.753274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 497/973] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:25.759159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.792681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.794683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.794683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.794683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.795119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.796659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.798569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.841065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.sql.core [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:36:25.841065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:36:25.841065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [parameters: {'user_id': 'd949ed275a9b424586f7bfc0ad023870', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:36:25.841065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application conn.query(q) May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result.read() May 25 14:36:25.853005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:36:25.854031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application next(self.gen) May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application yield resource May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application next(self.gen) May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise self.value May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application session.commit() May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:36:25.855010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.session.flush() May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application compat.raise_( May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:36:25.855918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:36:25.857339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application conn.query(q) May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application result.read() May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:36:25.858445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd949ed275a9b424586f7bfc0ad023870', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:25.866023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 14:36:25.867804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 477/974] 10.222.0.74 () {60 vars in 1131 bytes} [Wed May 25 14:36:25 2022] POST /identity/v3/users => generated 160 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:25.869852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.888059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9344-f0e5-465f-a238-45de611b780e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:25.927632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:25.928471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:25.928820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:25.929326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:25.930192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:25.930602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:25.932085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:25.938054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 498/975] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:25.944682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:25.988638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:25.996109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f314a-d192-4d01-98bb-453e93ca87b7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:26.040528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:26.041412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.041798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.042201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.050829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:26.052916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:26.070816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:26.072058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:36:26.073218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:26.079872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 478/976] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:26.081597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:26.085402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:26.096094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0dbdd-7098-4400-a2f4-994b1a0c976f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:26.135267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:26.136044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.136333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.136620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.137392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:26.137708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:26.139583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:26.150018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 499/977] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:26.151389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:26.217806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228a331-c54e-4cd7-9cd2-fb2ca50655dc None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:26.265316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:26.267117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.267596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.268113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.269058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:26.269808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:26.270844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907f77d-a43c-454f-b414-4a03d108f598 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:26.277028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6d46-57e7-4288-a998-5e750f80bfdb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:26.315346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:26.317312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.317312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.317312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.317562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:26.318874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:26.334855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 479/978] 10.222.0.74 () {56 vars in 1101 bytes} [Wed May 25 14:36:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:26.336951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48757-89a1-4f9e-9789-1b5f08193859 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.337249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48757-89a1-4f9e-9789-1b5f08193859 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.343622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48757-89a1-4f9e-9789-1b5f08193859 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.343622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 480/979] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:26.344983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10052e4c-a249-4833-bfe5-6b22b8da5afa None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.345277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10052e4c-a249-4833-bfe5-6b22b8da5afa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.345561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10052e4c-a249-4833-bfe5-6b22b8da5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.365323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-10052e4c-a249-4833-bfe5-6b22b8da5afa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:26.366932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-10052e4c-a249-4833-bfe5-6b22b8da5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:26.378431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:26.423571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 500/980] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:26.464111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 481/981] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:26.488068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:26.506294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:26.605321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7023c4-bf34-472e-a452-9569d93f5370 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:26.643823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:26.644491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:26.644773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:26.645079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:26.645769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:26.647165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bffb15-61fe-4dc8-bad3-fe48da69037d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:26.714142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 501/982] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:26 2022] POST /identity/v3/services => generated 242 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:27.854222 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1093-dd1a-401e-8e21-cb156b6cfb09 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:27.854902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1093-dd1a-401e-8e21-cb156b6cfb09 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:27.854902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1093-dd1a-401e-8e21-cb156b6cfb09 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:27.856342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 482/983] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:27.861391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:27.861912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:27.862099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:27.879286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:27.880475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:27.952249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:27.960514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 502/984] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:27.982488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c1093-dd1a-401e-8e21-cb156b6cfb09 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:28.085540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c1093-dd1a-401e-8e21-cb156b6cfb09 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:28.118854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:28.119558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.119841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.120117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] PATH_INFO: `/v3/services/image` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.120767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:28.122121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:28.134235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:28.135268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:28.136113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:28.138999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 483/985] 10.222.0.74 () {56 vars in 1105 bytes} [Wed May 25 14:36:27 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:28.141299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:28.239582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d154-f2af-402c-92ed-e9b5aca9c588 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:28.273842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:28.274505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.274829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.275108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.275933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] RBAC: Adding query filter params (name=image) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:28.276248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:28.277615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72954993-2fef-4470-bf52-fecda347f31d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:28.287023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 503/986] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:28.288518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:28.292314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:28.396188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee5c2-7c87-45b8-8964-f6d4ec8c7987 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:28.430623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:28.431331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.431615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.431896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.432598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:28.432910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:28.434272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e70675-91b6-46ea-aa97-7a2900c256c5 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:28.442859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 484/987] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:28.705620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbe91-6f01-4f61-b97e-8da4aacc2000 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.705815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbe91-6f01-4f61-b97e-8da4aacc2000 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.706095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbe91-6f01-4f61-b97e-8da4aacc2000 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.707653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 504/988] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:28.712869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63933bd-3113-4bd5-9f02-efc208d38eb8 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.713163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63933bd-3113-4bd5-9f02-efc208d38eb8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.713444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63933bd-3113-4bd5-9f02-efc208d38eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.730471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b63933bd-3113-4bd5-9f02-efc208d38eb8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:28.731694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b63933bd-3113-4bd5-9f02-efc208d38eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:28.801140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:28.807146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 485/989] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:28.829374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bbe91-6f01-4f61-b97e-8da4aacc2000 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:28.894039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:28.937455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bbe91-6f01-4f61-b97e-8da4aacc2000 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:28.947410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce990-04fe-4dc8-ab58-abc4abe8b8e1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.947902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce990-04fe-4dc8-ab58-abc4abe8b8e1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.947992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce990-04fe-4dc8-ab58-abc4abe8b8e1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.949518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 486/990] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:28.954246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.954543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.954860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.973076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:28.974277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:28.975309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:28.976303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:28.976583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:28.976858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:28.977515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:28.978903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:28.991345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:28.992364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:28.993987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 505/991] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:28.998428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.050842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 487/992] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:29.072645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.107938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e365af-2159-46fb-92d9-644c852b34fd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.135929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.144798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.145471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.145760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.150225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.150225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.150225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.150225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.157711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 506/993] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:29.158885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.189160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363381c-e3e8-4329-8299-c2b0602c6de2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.218515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.230605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.230605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.230605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.230605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] PATH_INFO: `/v3/services/compute` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.231048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.231301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:29.243186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:29.244225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:29.245103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 488/994] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:29.251011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.270214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a6fda-244d-41ba-8161-a943feeddf40 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.306174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.307132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.307417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.307692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.308557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.309180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.311031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.317751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 507/995] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:29.323555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.359627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe849160-ab8e-4b46-881a-8e4b13ed92af None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.385075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.395016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.395675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.397505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.397505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.397505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] RBAC: Adding query filter params (name=compute) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.397505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.398593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.404369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 489/996] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:29.411121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.432721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dfa1f-723a-4915-90de-17925dea13c3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.466570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.470558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.471788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.472077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.472359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.477406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] RBAC: Adding query filter params (interface=admin, service_id=337c6d612da94d5c99d3ea2ac9c3cdff, region_id=RegionOne) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.477406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.477406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.480744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 508/997] 10.222.0.74 () {56 vars in 1249 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/endpoints?service_id=337c6d612da94d5c99d3ea2ac9c3cdff&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:29.487719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.512421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6bee-a387-4bbc-a1f1-3978a966aeab None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.550912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.551597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.551878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.552156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.552876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] RBAC: Adding query filter params (type=compute) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.553187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.554596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.560121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 490/998] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:29.565826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.612778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93616a-5ea1-41f4-9183-b91f4943349f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.636647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.656664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.657333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.657624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.657905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.659193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.660573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.666795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 509/999] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:36:29.678556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0eb5-6099-4f99-b6cf-3a7624ca9a7f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.703226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:29.718721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.719387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.719669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.719945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.720791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] RBAC: Adding query filter params (interface=public, service_id=8121f1fdf0664a11a0838e0db80d03e2, region_id=RegionOne) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:29.721632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.723195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fb4c1-855a-4d51-99fd-6cb24f15fde4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.729368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 491/1000] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/endpoints?service_id=8121f1fdf0664a11a0838e0db80d03e2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:29.735091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:29.841233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f188-705c-4ff4-b7ea-6e75de70dc9e None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:29.876917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:29.878104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:29.878394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:29.878706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:29.879388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:29.880766 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74ac6d-9213-41e0-85cb-f2f11eb9fa3c None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:29.887266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 510/1001] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:29 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:36:30.653699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2dfb-e4eb-47c4-8bb8-5422c7e8e09e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:30.654203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2dfb-e4eb-47c4-8bb8-5422c7e8e09e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:30.654203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2dfb-e4eb-47c4-8bb8-5422c7e8e09e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:30.655718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 492/1002] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:30.661376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5d8f-d607-4508-b78f-f95723548021 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:30.661791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5d8f-d607-4508-b78f-f95723548021 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:30.661961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5d8f-d607-4508-b78f-f95723548021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:30.680150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c6ed5d8f-d607-4508-b78f-f95723548021 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:30.681426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c6ed5d8f-d607-4508-b78f-f95723548021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:30.723993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:30.748672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 511/1003] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:30.771113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2dfb-e4eb-47c4-8bb8-5422c7e8e09e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:30.874875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2dfb-e4eb-47c4-8bb8-5422c7e8e09e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:30.907916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:30.909051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:30.909337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:30.909613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:30.910343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:30.911783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671de34-bf52-492a-b8db-f33e6cd69987 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:31.001122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 493/1004] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:30 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:31.880547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043764de-effe-40cc-ab35-a73e597d5248 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:31.881065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043764de-effe-40cc-ab35-a73e597d5248 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:31.881172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043764de-effe-40cc-ab35-a73e597d5248 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:31.882491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 512/1005] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:31.887764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf6d41-61dd-4913-889c-fc08cbcfb1e5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:31.888194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf6d41-61dd-4913-889c-fc08cbcfb1e5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:31.888389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf6d41-61dd-4913-889c-fc08cbcfb1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:31.906779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-8fdf6d41-61dd-4913-889c-fc08cbcfb1e5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:31.908010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8fdf6d41-61dd-4913-889c-fc08cbcfb1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:32.004021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.036236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 494/1006] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:31 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:32.058630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043764de-effe-40cc-ab35-a73e597d5248 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.089685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263a78-5e5f-4600-b23a-927316090605 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.089984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263a78-5e5f-4600-b23a-927316090605 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.090272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263a78-5e5f-4600-b23a-927316090605 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.091694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 495/1007] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:32.097195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.097487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.097766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.115435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:32.116676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:32.167841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043764de-effe-40cc-ab35-a73e597d5248 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.192328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 496/1008] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:32 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:32.202388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.203234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.203523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.203803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.204458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.206393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.215081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:32.219006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:32.220015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:32.220891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 513/1009] 10.222.0.74 () {56 vars in 1126 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:32.226006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.277241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.324947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.327975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7c253-d3f2-4601-a78c-08078947b1c0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.342364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b92f6-8924-4a55-9f0f-b4ef2a392b9f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.363191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.364129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.364752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.365438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.366474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.368373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.377698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.378358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.378673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.378949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.379651 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:32.379962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:32.380499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:32.381490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:36:32.382364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:32.383157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.383236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 497/1010] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:32.388501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.396422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 514/1011] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:32.397239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.499518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250763b5-056f-4987-917a-6c3aaeb3b6da None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.500396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930b497-3584-465c-ae9c-89a6e4ebdb77 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.501798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.535626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.536292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.536575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.536856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.537559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:32.537872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.539283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.540116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.540778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.541228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.541509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.542461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:32.542853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.544226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.550477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 515/1012] 10.222.0.74 () {56 vars in 1122 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:32.551519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.554198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 498/1013] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:32.559718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.591130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.661255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c3ca8-db39-4793-ba61-9af89929ba67 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.664506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd4fc-b342-4f81-872f-864cb345e7b0 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.698892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.699840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.700301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.700754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.701284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.701896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.702002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.702217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.702495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.703234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:32.703545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.704290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf308f1-1598-48c2-8690-99fcc4101727 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.704921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:32.710032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 499/1014] 10.222.0.74 () {56 vars in 1112 bytes} [Wed May 25 14:36:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:32.715774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:32.794403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 516/1015] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:32.817272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:32.827711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627353f-255e-4aed-8724-ce4aa9c19d69 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:32.874091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:32.874801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:32.875090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:32.875367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:32.876180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:32.878317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60791ea8-2722-498f-a9a9-72f75206c9cc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:33.006117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 500/1016] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:36:33.238164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100c7bc-246d-4111-a2cd-c92980eef9f0 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.238706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100c7bc-246d-4111-a2cd-c92980eef9f0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.238814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100c7bc-246d-4111-a2cd-c92980eef9f0 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.240173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 517/1017] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:33.245501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.245793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.246072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.262948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:33.264139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:33.350685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 501/1018] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:33.369241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100c7bc-246d-4111-a2cd-c92980eef9f0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:33.398816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:33.476008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100c7bc-246d-4111-a2cd-c92980eef9f0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:33.512015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:33.512721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.513006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.513283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.513983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:33.515639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:33.527750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:33.528715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:33.529637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 518/1019] 10.222.0.74 () {56 vars in 1105 bytes} [Wed May 25 14:36:33 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:33.536529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:33.607361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:33.638939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbdb48-3665-418b-8f39-ed0697c87454 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:33.674847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:33.675515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.675950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.676237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.676941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] RBAC: Adding query filter params (name=image) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:33.677254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:33.678620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:33.688106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 502/1020] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:33.689321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:33.780827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:33.790822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59643a-e94e-4181-a2eb-1bd6e80b14f4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:33.824482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:33.825145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.825427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.825705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.826406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] RBAC: Adding query filter params (type=image) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:33.826741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:33.828106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:33.838100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 519/1021] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:33.839418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:33.944831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d3009-36bd-4356-b19e-77ffca3170fa None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:33.980187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:33.980845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:33.981129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:33.981403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:33.982193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] RBAC: Adding query filter params (interface=public, service_id=1e97209aa89844efba2b64302e0a17eb, region_id=RegionOne) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:33.982511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:33.983899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c863ef-1efd-434f-a5e7-6992c41fb987 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:33.985430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:33.994726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 503/1022] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:36:33 2022] GET /identity/v3/endpoints?service_id=1e97209aa89844efba2b64302e0a17eb&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:33.995670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:34.098373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa31-0918-4222-8606-2e9f6d65e496 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:34.132133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:34.132793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:34.133078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:34.133357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:34.134030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:34.135422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb20cb-e155-4b18-b2d7-37e58a1f06d6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:34.150920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 520/1023] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:36:33 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:36:35.037933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895199-7be8-4cd0-9886-ffa1c84bee7a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.038493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895199-7be8-4cd0-9886-ffa1c84bee7a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.038493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895199-7be8-4cd0-9886-ffa1c84bee7a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.040141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 504/1024] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:35.045559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9cf8b-2ea4-4d3a-8545-fcc6de69602b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.045996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9cf8b-2ea4-4d3a-8545-fcc6de69602b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.046164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9cf8b-2ea4-4d3a-8545-fcc6de69602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.063214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a3b9cf8b-2ea4-4d3a-8545-fcc6de69602b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:35.064400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a3b9cf8b-2ea4-4d3a-8545-fcc6de69602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:35.078923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:35.134891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 521/1025] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:35.158848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f895199-7be8-4cd0-9886-ffa1c84bee7a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:35.262765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f895199-7be8-4cd0-9886-ffa1c84bee7a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:35.297430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:35.298137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.298418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.298715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] PATH_INFO: `/v3/domains/default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.300462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:35.301876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:35.307889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 505/1026] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:35.310019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7c782-805e-4a4c-8578-cb93b511e7b8 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.310422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7c782-805e-4a4c-8578-cb93b511e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.310630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7c782-805e-4a4c-8578-cb93b511e7b8 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.311286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:35.312019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 522/1027] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:35.320183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.320479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.320763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.342486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:35.344378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:35.381652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:35.419919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912784f-67c1-4d49-a075-abfd2c0cfacb None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:35.421757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 523/1028] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:35.445669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:35.456431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:35.457408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.457857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.458257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.459271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:35.461236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:35.525655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:35.543717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 506/1029] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:36:35 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:35.545045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dec119-97ce-4577-a7de-346eacae0257 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:35.594613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:35.595385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.595722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.596530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.608983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:35.617652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:35.618711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:35.624762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 524/1030] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:35.627085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:35.739694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85708fa9-127c-4d99-99ce-30219e2b5b08 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:35.757516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:35.776572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:35.777248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.777532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.777812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.778565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:35.778940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:35.780311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:35.790900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 507/1031] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:35.796614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:35.898855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a7cf-2e70-4931-a61a-128fa84034fb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:35.932088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:35.932887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:35.933161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:35.933436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:35.942843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:35.945516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:35.947003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:35.965094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:35.966027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:35.967364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:35.967364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:35.967364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:35.967364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:35.967836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 525/1032] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:35.972466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.071705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90b9e7-ac38-4f41-94e6-52233a25a642 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.105998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.106673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.106961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.107238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.107952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:36.108261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.109621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.121162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 508/1033] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:36.127399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.234147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc9d21-cce6-4910-9b77-2e8e245e23f9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.268375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.269172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.269462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.269743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.277717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.279205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-085e5d38-2f80-4e64-beff-0f9f1044fb16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:36.293351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:36.294326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:36.295408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:36.299514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 526/1034] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:36.300833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.348634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ddf62-0cea-4408-a56c-7f5ba1d67431 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.348950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ddf62-0cea-4408-a56c-7f5ba1d67431 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.349195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ddf62-0cea-4408-a56c-7f5ba1d67431 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.351196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 527/1035] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:36.351809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.355971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.356264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.356545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.373841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:36.375176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:36.423088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d88b-e1f7-4f4b-879f-9eb17fe049db None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.426671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.449372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 528/1036] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:36.460668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.461340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.461629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.461907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.462769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:36.463087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.464503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.472230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.488936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 509/1037] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:36.490034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.583277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836327b1-02df-4c72-b795-8cdbb7ef0579 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.590134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.593930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfc915-3317-4c10-b4c7-50c70c06751d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.620140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.621135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.621593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.622022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] PATH_INFO: `/v3/services/image` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.622775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.624218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.628698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.629337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.629618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.629891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:36.636449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:36.637410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:36.638252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 529/1038] 10.222.0.74 () {56 vars in 1105 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:36.643541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.690482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.704949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.706403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.745990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75a3a-27ff-4c1b-b7ed-eb1cac498136 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.787491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.788516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.788806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.789083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.790099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] RBAC: Adding query filter params (name=image) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:36.790566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.791966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.797730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 530/1039] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:36.804186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.809000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.813924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 510/1040] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:36.819567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.919845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf42cb3-2dc6-465d-a38a-488ddf6ef738 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.921057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae20ea7-5648-451d-ae52-8d4663d93dc9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:36.931353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:36.955518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.958819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.958819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.958819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.959051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:36.959146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] RBAC: Adding query filter params (type=image) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:36.959146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.959287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:36.959287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:36.959287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:36.959462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.964756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 531/1041] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:36.970252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:36.970484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:36.972442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:36.991644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:36.992620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 511/1042] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:36.998420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:37.048497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:37.079790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974981-83bb-4f7c-9df2-722337606468 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:37.102019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8d105-9904-4984-91c5-1009da1ce9f8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:37.114740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:37.115401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.115681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.115958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] PATH_INFO: `/v3/endpoints` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.116700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:37.118147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:37.141226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:37.141884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.142171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.143001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.143726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:37.144038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:37.145388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:37.156440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 512/1043] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:37.162328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:37.189270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 532/1044] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:36:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:36:37.270423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52600b69-5a46-49cf-bcd5-e0e7534fa392 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:37.281787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:37.319132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:37.319948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.320239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.320516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.332063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:37.333797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:37.349681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:37.350948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:37.351879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 513/1045] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:37.358232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:37.394029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:37.465597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958a55b-b1e7-426d-8ad5-3fc7f60bbedb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:37.499080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:37.499747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.500034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.500311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.501094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:37.501413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:37.503423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf1019-08c3-440f-b78b-370d03128676 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:37.514802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 533/1046] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:37.521371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:37.626321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5285c7-a71e-4913-9620-9995066f5895 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:37.663620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:37.664367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.664655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.664935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.665790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] RBAC: Adding query filter params (role.id=d4196859f6d445d990d30580d822baec, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:37.666112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:37.667778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b932a9c-ffc7-4392-aaa5-d47722bab76d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:37.682676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 514/1047] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:37 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d4196859f6d445d990d30580d822baec => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:37.848280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49e192-fa18-4b69-adf0-de30a39e45f1 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.848598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49e192-fa18-4b69-adf0-de30a39e45f1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.848886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49e192-fa18-4b69-adf0-de30a39e45f1 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.850282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 534/1048] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:37.856106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:37.856401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:37.856686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:37.874109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:37.875345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:37.899756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:37.944943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 515/1049] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:37.970137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49e192-fa18-4b69-adf0-de30a39e45f1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:38.065245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:38.075607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49e192-fa18-4b69-adf0-de30a39e45f1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:38.109767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:38.110482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:38.110798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:38.111079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:38.112776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:38.114205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16376f-34ca-4b1d-ac6c-45e064942401 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:38.120487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 535/1050] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:38.121947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:38.221294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15c36-cf09-4cd3-a9f6-4797de23f633 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:38.254757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:38.255431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:38.255717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:38.255997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:38.256718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:38.258137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be265971-8d25-4a47-8a6b-df2acbf96f79 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:38.374753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 516/1051] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:36:38 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:39.426445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7be1-f6ba-4448-a4ef-d2786277c62e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:39.426979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7be1-f6ba-4448-a4ef-d2786277c62e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:39.427089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7be1-f6ba-4448-a4ef-d2786277c62e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:39.428415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 536/1052] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:39.433559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:39.433953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:39.434142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:39.450945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:39.452120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:39.463700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:39.520648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 517/1053] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:39.543262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b7be1-f6ba-4448-a4ef-d2786277c62e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:39.698257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b7be1-f6ba-4448-a4ef-d2786277c62e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:39.754374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:39.755527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:39.756035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:39.756527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:39.800604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:39.869308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:39.872032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fea28-d641-4100-a5e6-abbe3c1f2895 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:39.982753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 537/1054] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:39.985794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.013898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74a530-b858-4f24-a694-9cf5e0591516 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.014254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74a530-b858-4f24-a694-9cf5e0591516 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.014481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74a530-b858-4f24-a694-9cf5e0591516 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.016022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 538/1055] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:40.021467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.021760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.022042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.038371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:40.039594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:40.091271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4e9c-8daa-4001-9503-af01f0c411cb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.093384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.107187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 539/1056] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:40.127763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.128563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.128849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.129405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.130365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.139693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.141141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.148367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:40.160177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:40.166600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 518/1057] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:40.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.238140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e8336-114f-4ec2-a99e-78c4053a1f86 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.270276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14617b-cdd3-4c02-b430-c0663989d38e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.272719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.273533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.273817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.274095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.284450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:40.292593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:40.293514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:40.294454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:40.299295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 540/1058] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:40.300079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.308667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.309332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.309618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.309896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.310990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:40.311094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.312465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.338714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 519/1059] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:40.364512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.420970 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79095ae3-3d0a-4277-8b49-1eff01856578 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.461246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.462072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.462262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.462546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.463339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:40.463655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.465055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4292cc-f983-4c0e-aace-4ad9cc3ce2d0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.479042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 541/1060] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:40.481027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.546964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.589802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffca1f-8ff2-4b17-92e8-ce2449f0f927 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.623027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785930c-8b00-4bc7-86db-cbc9d6eb23ed None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.623319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785930c-8b00-4bc7-86db-cbc9d6eb23ed None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.623682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785930c-8b00-4bc7-86db-cbc9d6eb23ed None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.625214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 542/1061] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:40.626315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.627186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.627478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.627756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.630327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.630926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.630926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.640296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.641741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.648042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:40.649223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:40.661953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:40.662908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:40.663835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:40.663835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:40.663835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:40.663835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:40.663835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 520/1062] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:40.668976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.721300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 543/1063] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:40.744792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.768552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7a43d-75a3-4dd7-8b48-97d5ab3cb82e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.778703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.785399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:40.801829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.802427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.802741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.803021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.803743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:40.804055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.805414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.817566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 521/1064] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:40.822333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.840287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a9cd-1a77-45bf-91c7-e983b8933da9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.874907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.875694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.875996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.876268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.886017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:40.894042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:40.895112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:40.896097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 544/1065] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:40.900993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:40.922974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973cea-5847-4efb-b9a5-67738660cf83 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:40.956626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:40.957408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:40.957739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:40.958020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:40.965744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:40.967242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:40.982183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:40.983183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:40.989019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 522/1066] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:40.989621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.001299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.004426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b9db1-6bd4-488a-95a7-24e2ec4451cf None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.019519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.038955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.039617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.039897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.040173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.040907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:41.041218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.042626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.050682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 545/1067] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:41.056399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.089955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8cad0-8ba7-4d6b-ba27-9cf1e2b3a005 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.124603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.125267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.125552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.125830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.127176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:41.127496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.128919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.146066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 523/1068] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:36:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:41.151686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.153350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac50b5a-0912-424c-91d6-1393f56c437d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.185907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.186813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.187103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.187381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] PATH_INFO: `/v3/users/d2cc679211074fa4a189bae8b44919f9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.200702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] RBAC: Authorizing `identity:get_user(user_id=d2cc679211074fa4a189bae8b44919f9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.202152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.203980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 546/1069] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/users/d2cc679211074fa4a189bae8b44919f9 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:41.210212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.230353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.236302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.247857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9113e-7175-4852-b3ae-66e6633a930e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.282345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.283193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.283476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.283750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.293935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.295405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.306389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec953-3240-44f5-b659-0e11f8f422c7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:41.313969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:41.315490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:41.316393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:41.316393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:41.316393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:41.316393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:41.316393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 524/1070] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:41.320614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.348989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.349645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.349929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.350206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.416017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.417464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.421137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892ccf-57c9-4608-bd13-f686026c2aae None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.437703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.454769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.455406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.455689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.455969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.456687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:41.457005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.458647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.466772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.470071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 525/1071] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:41.476131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.514936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 547/1072] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:36:41.521093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.572780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03351-894f-41cc-b40a-90080cfac4fe None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.607337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.608124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.608407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.608686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.615245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.616669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.618518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81f1b3-63db-4b0c-add3-82f680c840ae None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:41.630609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:41.631596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:41.632875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 526/1073] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:41.637504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.639506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.653528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.654372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.654678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.654960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.656127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.657557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.659884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 548/1074] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:41.665912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.696665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.737208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793681a-81da-4f8d-9a52-4cedf9dcf06a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.762562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a063501-156a-4903-b260-f860e64e2960 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.770407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.771092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.771379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.771661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.772445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:41.772765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.774188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.785923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 527/1075] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:41.791735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:41.796607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.797256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.797539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.797817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.798655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=df8b53bfcb2d43a682b59f30962f25d8, user.id=d2cc679211074fa4a189bae8b44919f9) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:41.798983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.800414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce9a4-bf47-45b4-8004-26af148cbae0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:41.805949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 549/1076] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:41 2022] GET /identity/v3/role_assignments?user.id=d2cc679211074fa4a189bae8b44919f9&scope.project.id=df8b53bfcb2d43a682b59f30962f25d8&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:41.896191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d64d0-25b5-4646-9e27-504d68d30df9 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:41.916890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:41.930937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:41.931838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:41.932119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:41.932394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/d4196859f6d445d990d30580d822baec` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:41.934711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=d4beeef3c39c44f39c8c7ee5f13e9623, role_id=d4196859f6d445d990d30580d822baec)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:41.936189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25f82a-ca9b-4591-b280-cbfaedeeb296 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:42.023137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 528/1077] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:41 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/d4beeef3c39c44f39c8c7ee5f13e9623/roles/d4196859f6d445d990d30580d822baec => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:36:43.963867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3ca5-9f6d-4deb-abb6-3322f4301b6f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:43.964415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3ca5-9f6d-4deb-abb6-3322f4301b6f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:43.964415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3ca5-9f6d-4deb-abb6-3322f4301b6f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:43.965767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 550/1078] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:43.971661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcce1-7a48-4e82-a551-1cadc8b4e037 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:43.972042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcce1-7a48-4e82-a551-1cadc8b4e037 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:43.972225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfcce1-7a48-4e82-a551-1cadc8b4e037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:43.988578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6adfcce1-7a48-4e82-a551-1cadc8b4e037 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:43.989736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6adfcce1-7a48-4e82-a551-1cadc8b4e037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:44.058210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 529/1079] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:44.080800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f3ca5-9f6d-4deb-abb6-3322f4301b6f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.133866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.181367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f3ca5-9f6d-4deb-abb6-3322f4301b6f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.202260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a89418-f340-4959-9780-ff241a74c3b1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.202546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a89418-f340-4959-9780-ff241a74c3b1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.202872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a89418-f340-4959-9780-ff241a74c3b1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.204233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 530/1080] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:36:44.209972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.210260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.210540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.216374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.217113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.217400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.217681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.227262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:44.227631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:44.228455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:44.235309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:44.236387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:44.237239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 551/1081] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:44.242306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.293870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 531/1082] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:44.318041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.341337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcdbc9-f614-4a13-bb7c-c3724b101cf3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.353911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.363149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.374340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.375045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.375331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.375614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.376365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:44.377213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.378613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.386971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 552/1083] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:44.393408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.418819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60a1e-056e-4e4a-a4cf-8c1a3b452859 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.452016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.452823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.453107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.453385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.464693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:44.472938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:44.473877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:44.475152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 532/1084] 10.222.0.74 () {56 vars in 1120 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:44.479970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.489899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282af8-2561-4cfa-85b5-22aed469777a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.528318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.529185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.529466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.529741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] PATH_INFO: `/v3/users/d2cc679211074fa4a189bae8b44919f9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.531006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] RBAC: Authorizing `identity:get_user(user_id=d2cc679211074fa4a189bae8b44919f9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.532441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.534053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 553/1085] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/users/d2cc679211074fa4a189bae8b44919f9 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:44.539608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.581048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.581715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.584532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5324b-4434-4b1d-ac41-03d89ae84dfe None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.617898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.618563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.618874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.619153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.619951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:44.620263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.621622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.630120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 533/1086] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:44.635677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.641167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b51b7-55fa-46f9-a8b1-46ec374b57f7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.675834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.676690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.676974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.677248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.678520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.680027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.681704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 554/1087] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:44.687444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.732545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4777c-ddb0-441d-8757-e52f7d9a4cf7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.767684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.768466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.768750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.769030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.779442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.780878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.784556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2badf81-28b5-47b5-9dbf-b4f70f413471 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.794920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:44.797538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:44.799148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:44.800088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:44.800088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:44.800088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:44.800088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:44.800088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 534/1088] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:44.804878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:44.820586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:44.821688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.822568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.822883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.823163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8/users/d2cc679211074fa4a189bae8b44919f9/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.825203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] RBAC: Authorizing `identity:create_grant(project_id=df8b53bfcb2d43a682b59f30962f25d8, user_id=d2cc679211074fa4a189bae8b44919f9, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.826658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.896136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 555/1089] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:44 2022] PUT /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8/users/d2cc679211074fa4a189bae8b44919f9/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:36:44.900253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a36bce-c7e0-470f-8232-c67de0f8ce67 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:44.939309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:44.939989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:44.940351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:44.940647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:44.941384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:44.941695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:44.943179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:44.956401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 535/1090] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:36:44.962403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:45.067231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e69e65-a3e7-4a98-a77f-1460e0ca5955 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:45.103657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:45.106258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:45.106258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:45.106258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:45.112669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:45.114105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:36:45.128356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:45.129466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:45.130527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 556/1091] 10.222.0.74 () {56 vars in 1093 bytes} [Wed May 25 14:36:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:36:45.135281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:45.259540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:45.284409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eece13-32bb-4f6b-b135-324bc875473f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:45.317623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:45.318215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:45.318498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:45.318804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:45.319642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:45.319956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:45.321381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:45.339302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 536/1092] 10.222.0.74 () {56 vars in 1133 bytes} [Wed May 25 14:36:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:45.345014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:45.374520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:45.443121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23143b9-ca1b-4e44-b0c4-4591463a93e2 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:45.477629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:45.478272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:45.478558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:45.478870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:45.543533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:45.544984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:45.572493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:45.641969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 557/1093] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:45.648398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:45.744202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1045712-e837-4b1d-856d-bdf4c4d54675 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:45.773054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:45.777762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:45.778537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:45.778846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:45.779126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:45.788673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:45.790385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:36:45.806986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:45.807949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:45.808840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:36:45.808840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:36:45.808840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:36:45.808840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:45.808840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 537/1094] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:45.813756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:45.911004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151ba8-3d2a-4ca8-95f1-16fd7668d656 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:45.938089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:45.943725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:45.944387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:45.944670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:45.944996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:45.945661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:45.945971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:45.947374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:45.959090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 558/1095] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:45.964547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:46.063410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4efc0-12f2-4c65-8919-6bea8bdd5ce6 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:46.097023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:46.097812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:46.098093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:46.098365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:46.104875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:46.106291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:36:46.119521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:46.120557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:36:46.121788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 538/1096] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:46.126445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:46.223714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501346b-d07e-42f3-8df4-dc3baf37dea0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:46.255877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:46.256529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:46.256816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:46.257095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:46.257872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:46.258185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:46.259652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ecdfc-2ca3-496a-a61d-9f882d5a9289 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:46.271468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 559/1097] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:46.277296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:46.322592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:46.377502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0628e-cdb0-4658-baf3-7ac39ff3f6df None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:46.410628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:46.411314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:46.411600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:46.411880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:46.412709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] RBAC: Adding query filter params (role.id=d4196859f6d445d990d30580d822baec, scope.project.id=c07159d21300478e88a07006614fdded, user.id=d4beeef3c39c44f39c8c7ee5f13e9623) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:46.413026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:46.414450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b6142-dc9e-4ed5-b808-882fbf928239 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:46.420286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 539/1098] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:46 2022] GET /identity/v3/role_assignments?user.id=d4beeef3c39c44f39c8c7ee5f13e9623&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d4196859f6d445d990d30580d822baec => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:47.123855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d297d65-0373-4d52-82df-9a7973941570 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.124158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d297d65-0373-4d52-82df-9a7973941570 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.124498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d297d65-0373-4d52-82df-9a7973941570 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.125867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 560/1099] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:47.132240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.132540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.132819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.149965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:47.151196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:47.218959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 540/1100] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:47.243031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d297d65-0373-4d52-82df-9a7973941570 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:47.294105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:47.344018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d297d65-0373-4d52-82df-9a7973941570 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:47.376918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:47.377645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.377932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.378211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.387947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:36:47.395580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:36:47.396737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:36:47.397665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 561/1101] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:36:47.402715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:47.441140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:47.499045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ddef0-724d-4605-a138-224327d5f4f4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:47.531973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:47.532621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.532930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.533210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.533946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:47.534257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:47.535638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:47.543401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 541/1102] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:36:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:47.549507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:47.646292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721efc80-f9f4-427b-8654-c1f4809c58c1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:47.656047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:47.679029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:47.679888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.680175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.680455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] PATH_INFO: `/v3/users/d2cc679211074fa4a189bae8b44919f9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.681631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] RBAC: Authorizing `identity:get_user(user_id=d2cc679211074fa4a189bae8b44919f9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:47.683091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:47.684971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 562/1103] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:47 2022] GET /identity/v3/users/d2cc679211074fa4a189bae8b44919f9 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:47.690998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:47.784941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:47.787741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e93e-e809-4e17-ba47-7509b379f358 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:47.819962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:47.820600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:47.820886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:47.821192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:47.885782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:47.887250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:47.977743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:47.981287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 542/1104] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:47.987177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:48.083820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ed8f8-9ba7-4d46-b264-ba24cf19402f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:48.115262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:48.116102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:48.116388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:48.116665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:48.117841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:48.119300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8bc8d-2ed7-463b-9fdf-4c324e419b52 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:48.120956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 563/1105] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:47 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:48.127434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:48.223186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a77ced-9023-4ece-a7d3-f122adf47a51 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:48.255852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:48.256427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:48.256714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:48.256995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:48.257812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=df8b53bfcb2d43a682b59f30962f25d8, user.id=d2cc679211074fa4a189bae8b44919f9) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:48.258128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:48.259575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465585-3280-448f-8da0-5d48c43a1750 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:48.265174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 543/1106] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:48 2022] GET /identity/v3/role_assignments?user.id=d2cc679211074fa4a189bae8b44919f9&scope.project.id=df8b53bfcb2d43a682b59f30962f25d8&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:50.452551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daa81b-b205-426a-8853-d5215c35c8df None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:50.453056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daa81b-b205-426a-8853-d5215c35c8df None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:50.453156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daa81b-b205-426a-8853-d5215c35c8df None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:50.454460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 564/1107] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:50.460626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:50.461044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:50.461156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:50.477440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:50.478601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:50.535216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:50.543064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 544/1108] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:50.568502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daa81b-b205-426a-8853-d5215c35c8df None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:50.614692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:50.666602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daa81b-b205-426a-8853-d5215c35c8df None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:50.700184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:50.700954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:50.701239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:50.701518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:50.702696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:50.704122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd7123-f33d-4cb1-976f-625bba262861 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:50.705726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 565/1109] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:50.711755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:50.805730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8c407-1881-46c3-927a-cc3e5c390fc0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:50.838464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:50.839137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:50.839422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:50.839697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:50.840397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:50.841817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53c09b-e5b4-4225-8ed4-6af0d2a216cb None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:50.936784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 545/1110] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:36:50 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:53.157363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7b4b1-6bc8-4486-8d2b-fb3c3715e3ca None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.157921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7b4b1-6bc8-4486-8d2b-fb3c3715e3ca None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.157921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7b4b1-6bc8-4486-8d2b-fb3c3715e3ca None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.159292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 566/1111] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:53.164687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.165090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.165256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.181710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:53.182920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:53.228148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:53.247568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 546/1112] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:53.271521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7b4b1-6bc8-4486-8d2b-fb3c3715e3ca None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:53.369306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7b4b1-6bc8-4486-8d2b-fb3c3715e3ca None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:53.382999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:53.402246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:53.403183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.403466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.403739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] PATH_INFO: `/v3/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.405185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] RBAC: Authorizing `identity:get_role(role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:53.406544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:53.408575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 567/1113] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:53 2022] GET /identity/v3/roles/417b7981c02647a98c1d113122c9bdad => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:53.414266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:53.510114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e794fb0-34f8-466e-bbd4-6ba3ce514717 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:53.543185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:53.544037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.544319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.544594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] PATH_INFO: `/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.557238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=f9ca7bf5e3b84237a0a12b3b3756914f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:53.558688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:53.560303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 547/1114] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:53 2022] GET /identity/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:53.565843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:53.661119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f01e5-b4e2-4629-b8b9-3ac0b6cd62af None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:53.693048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:53.693671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.693953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.694226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.740927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:53.758211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:53.759672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:53.853040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 568/1115] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:53.858852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:53.898193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:53.955625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc14570-f4a3-4c25-a01d-ec2cd5930ab3 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:53.987791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:53.988637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:53.988921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:53.989201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:53.990354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:53.991806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9626a95-9ce4-4c73-bb00-e8dadced2ae6 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:53.993465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 548/1116] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:53 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:53.999363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:54.076577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:54.094342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc08736-ad90-40da-9b3c-00b10b05051a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:54.127314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:54.127949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:54.128234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:54.128509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:54.129325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=df8b53bfcb2d43a682b59f30962f25d8, user.id=f9ca7bf5e3b84237a0a12b3b3756914f) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:54.129639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:54.131091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980ed1b-1610-4b19-a1dc-0e5800857d36 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:54.136290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 569/1117] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:53 2022] GET /identity/v3/role_assignments?user.id=f9ca7bf5e3b84237a0a12b3b3756914f&scope.project.id=df8b53bfcb2d43a682b59f30962f25d8&role.id=417b7981c02647a98c1d113122c9bdad => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:56.267943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7c004-e4ac-4df3-a950-a53e49d44181 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.268498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7c004-e4ac-4df3-a950-a53e49d44181 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.268498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7c004-e4ac-4df3-a950-a53e49d44181 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.269856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 549/1118] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:56.275132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.275533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.275746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.291924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:56.293099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:56.359841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 570/1119] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:36:56.382194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7c004-e4ac-4df3-a950-a53e49d44181 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:56.479167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7c004-e4ac-4df3-a950-a53e49d44181 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:56.486212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:56.511591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:56.512536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.512815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.513091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] PATH_INFO: `/v3/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.514539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] RBAC: Authorizing `identity:get_role(role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:56.515932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:56.517870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 550/1120] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:56 2022] GET /identity/v3/roles/417b7981c02647a98c1d113122c9bdad => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:56.523474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:56.596236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:56.618278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f6f90-9b0f-4b83-9ab5-7998a6593db2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:56.651309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:56.652161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.652447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.652724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] PATH_INFO: `/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.653901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] RBAC: Authorizing `identity:get_user(user_id=f9ca7bf5e3b84237a0a12b3b3756914f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:56.655376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:56.656987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 571/1121] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:56 2022] GET /identity/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:56.662616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:56.759024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8089c04-d595-4ff4-b5ac-62ddb0ffb68c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:56.790482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:56.791306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.791584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.791855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.793009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:56.794428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330d397-6ba9-4172-a2dc-f941ff495e39 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:56.796100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 551/1122] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:56 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:56.802201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:56.897453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b59cc-12b9-41c2-89e4-1cbf691db4fb None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:56.930278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:56.931174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:56.931461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:56.931741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8/users/f9ca7bf5e3b84237a0a12b3b3756914f/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:56.933920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] RBAC: Authorizing `identity:create_grant(project_id=df8b53bfcb2d43a682b59f30962f25d8, user_id=f9ca7bf5e3b84237a0a12b3b3756914f, role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:56.935379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7274fe-7187-47c4-a78f-3bce9dc06653 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:57.008321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 572/1123] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:36:56 2022] PUT /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8/users/f9ca7bf5e3b84237a0a12b3b3756914f/roles/417b7981c02647a98c1d113122c9bdad => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:59.148471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2989-c7ee-4bd4-85d1-d271fd9d0503 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.149007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2989-c7ee-4bd4-85d1-d271fd9d0503 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.149007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2989-c7ee-4bd4-85d1-d271fd9d0503 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.150384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 552/1124] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:59.155660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.156091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.156217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.162578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:59.172902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:59.174727 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:59.238554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 573/1125] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:36:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:36:59.261797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b2989-c7ee-4bd4-85d1-d271fd9d0503 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:59.305009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:59.360654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b2989-c7ee-4bd4-85d1-d271fd9d0503 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:59.392748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:59.393584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.393867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.394136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] PATH_INFO: `/v3/roles/417b7981c02647a98c1d113122c9bdad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.395621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] RBAC: Authorizing `identity:get_role(role_id=417b7981c02647a98c1d113122c9bdad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:59.396968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:59.398322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 553/1126] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:59 2022] GET /identity/v3/roles/417b7981c02647a98c1d113122c9bdad => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:59.404132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:59.499859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83eb3b-7c3b-4cad-b5f7-df5d551341ff None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:59.532955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:59.533817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.534104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.534384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] PATH_INFO: `/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.535598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] RBAC: Authorizing `identity:get_user(user_id=f9ca7bf5e3b84237a0a12b3b3756914f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:59.537027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:59.538611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 574/1127] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:36:59 2022] GET /identity/v3/users/f9ca7bf5e3b84237a0a12b3b3756914f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:59.544039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:59.638126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11251c-2b89-4f06-8fba-e5b4f8568f1f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:59.648775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:59.671145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:59.671708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.671989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.672265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.735867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:59.737279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:59.828808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 554/1128] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:36:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:36:59.835327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:59.838143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:59.931690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246808-1211-4deb-98d4-1292a731aeff None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:59.964445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:59.965318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:59.965604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:59.965877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] PATH_INFO: `/v3/projects/df8b53bfcb2d43a682b59f30962f25d8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:59.967077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] RBAC: Authorizing `identity:get_project(project_id=df8b53bfcb2d43a682b59f30962f25d8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:59.968502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33f6dc-5c17-4605-9ad0-89096a008061 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:59.970149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 575/1129] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:36:59 2022] GET /identity/v3/projects/df8b53bfcb2d43a682b59f30962f25d8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:59.975956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:59.987049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:00.069667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14545998-cee8-4bf0-bdfd-50fe5902923c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:00.102389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:00.103046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:00.103329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:00.103605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:00.104420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] RBAC: Adding query filter params (role.id=417b7981c02647a98c1d113122c9bdad, scope.project.id=df8b53bfcb2d43a682b59f30962f25d8, user.id=f9ca7bf5e3b84237a0a12b3b3756914f) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:00.104734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:00.106136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9034b5-8291-47b9-9820-3a0466701b04 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:00.112351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 555/1130] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:36:59 2022] GET /identity/v3/role_assignments?user.id=f9ca7bf5e3b84237a0a12b3b3756914f&scope.project.id=df8b53bfcb2d43a682b59f30962f25d8&role.id=417b7981c02647a98c1d113122c9bdad => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:02.293508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989834a-7bff-4f5d-87a6-23cbae781c5e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:02.294032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989834a-7bff-4f5d-87a6-23cbae781c5e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:02.294148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989834a-7bff-4f5d-87a6-23cbae781c5e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:02.295459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 576/1131] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:02.301529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:02.301944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:02.302084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:02.318568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:02.319769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:02.384843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 556/1132] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:02.408280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989834a-7bff-4f5d-87a6-23cbae781c5e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:02.476288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:02.506700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989834a-7bff-4f5d-87a6-23cbae781c5e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:02.540734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:02.541504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:02.541786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:02.542060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:02.543239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:02.544713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ebaf-6b92-4c78-87ef-db02fb9c8495 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:02.546237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 577/1133] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:02.551898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:02.554937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:02.647738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea0c12-1115-4bdf-957c-9c686613bcff None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:02.680477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:02.681118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:02.681401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:02.681680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:02.682387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:02.683839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b207f-4df0-448c-a4ae-ade4b904bcbe None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:02.793588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 557/1134] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:37:02 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:37:04.983981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b258b86-c70b-4b16-8d1a-cede1c9a5bf2 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:04.984520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b258b86-c70b-4b16-8d1a-cede1c9a5bf2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:04.984520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b258b86-c70b-4b16-8d1a-cede1c9a5bf2 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:04.986393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 578/1135] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:04.991572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:04.991959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:04.992139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:05.009007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:05.010180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:05.058159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:05.074322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 558/1136] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:05.099100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b258b86-c70b-4b16-8d1a-cede1c9a5bf2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:05.197415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b258b86-c70b-4b16-8d1a-cede1c9a5bf2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:05.229698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:05.230469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:05.230778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:05.231058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:05.232216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:05.233668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845f46-41bb-4a92-abc9-6d8b5dfd3b7a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:05.235379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 579/1137] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:05.241474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:05.335712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd258417-c201-4a8d-a35b-0fd9038f6455 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:05.368561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:05.369213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:05.369497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:05.369775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:05.370480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:05.371987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a29e54-b3c7-4aac-9456-efcf37a0a2ae None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:05.465644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 559/1138] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:37:05 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:37:07.680250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5010491-0cc3-42ba-af32-11c263d6276b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:07.680762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5010491-0cc3-42ba-af32-11c263d6276b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:07.680762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5010491-0cc3-42ba-af32-11c263d6276b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:07.682096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 580/1139] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:07.687301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:07.687708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:07.687869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:07.704166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:07.705344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:07.740052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:07.769968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 560/1140] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:07.793241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5010491-0cc3-42ba-af32-11c263d6276b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:07.802127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:07.890263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5010491-0cc3-42ba-af32-11c263d6276b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:07.922950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:07.923720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:07.924003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:07.924282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:07.934214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:07.941879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:07.944190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:07.945241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 581/1141] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:07.949057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:08.048424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffe151-99f1-4cc7-87d2-90836d16bc91 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:08.081096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:08.081754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:08.082039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:08.082317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:08.083353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:08.083665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:08.085027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:08.092844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 561/1142] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:08.099158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:08.161852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:08.195413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c948-6d17-4aff-9f3b-628b6031d965 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:08.227683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:08.228527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:08.228812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:08.229091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] PATH_INFO: `/v3/users/75af96d2691041249e24e2fa646e4c43` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:08.241707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] RBAC: Authorizing `identity:get_user(user_id=75af96d2691041249e24e2fa646e4c43)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:08.243192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:08.244785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 582/1143] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:08 2022] GET /identity/v3/users/75af96d2691041249e24e2fa646e4c43 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:08.250318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:08.284208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:08.348867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6daa5b6-77c3-445c-aafc-51b22417ad26 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:08.381471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:08.382107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:08.382395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:08.382694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:08.446168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:08.447565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:08.478455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:08.540907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 562/1144] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:37:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:37:08.546778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:08.641808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72ea7c-9a11-42d6-bbea-2e9ce4d4e255 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:08.663686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:08.674159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:08.675014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:08.675296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:08.675570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] PATH_INFO: `/v3/projects/9b5cc7402cba44f0b26e551247d0e961` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:08.676730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] RBAC: Authorizing `identity:get_project(project_id=9b5cc7402cba44f0b26e551247d0e961)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:08.678158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299c761-8c65-424c-ae77-40e85c560e74 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:08.679829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 583/1145] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:08 2022] GET /identity/v3/projects/9b5cc7402cba44f0b26e551247d0e961 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:08.685737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:08.781692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace082ce-96a0-4f77-9964-89a769ad6e6b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:08.814376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:08.815038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:08.815323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:08.815601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:08.816422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=9b5cc7402cba44f0b26e551247d0e961, user.id=75af96d2691041249e24e2fa646e4c43) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:08.816740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:08.818155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f13a12-3956-4196-9edd-849858193275 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:08.819964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:08.823991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 563/1146] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:37:08 2022] GET /identity/v3/role_assignments?user.id=75af96d2691041249e24e2fa646e4c43&scope.project.id=9b5cc7402cba44f0b26e551247d0e961&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:10.966546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e499-9921-4328-8e5f-4434c9070cc6 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:10.967118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e499-9921-4328-8e5f-4434c9070cc6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:10.967118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e499-9921-4328-8e5f-4434c9070cc6 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:10.968482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 584/1147] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:10.974290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:10.974750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:10.974879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:10.991195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:10.992375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:11.055687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 564/1148] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:11.077243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e499-9921-4328-8e5f-4434c9070cc6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:11.174532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e499-9921-4328-8e5f-4434c9070cc6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:11.207255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:11.208281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:11.208561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:11.208832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:11.218441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:11.226061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:11.227172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:11.228400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 585/1149] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:11.233200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:11.327431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fec85-f95b-45fd-a80b-e632f769cc0e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:11.353398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:11.360875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:11.371785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:11.372264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 565/1150] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:11.378577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:11.431078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:11.476204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144679bd-d117-4254-96ab-c2e11c100c8d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:11.508167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:11.509017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:11.509303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:11.509583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] PATH_INFO: `/v3/users/75af96d2691041249e24e2fa646e4c43` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:11.510786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] RBAC: Authorizing `identity:get_user(user_id=75af96d2691041249e24e2fa646e4c43)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:11.512217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:11.513798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 586/1151] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:11 2022] GET /identity/v3/users/75af96d2691041249e24e2fa646e4c43 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:11.519872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:11.616278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db7b8-fb49-4eaa-a689-4b043cf4efb7 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:11.648155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:11.648992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:11.649282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:11.649557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] PATH_INFO: `/v3/projects/9b5cc7402cba44f0b26e551247d0e961` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:11.650747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] RBAC: Authorizing `identity:get_project(project_id=9b5cc7402cba44f0b26e551247d0e961)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:11.652166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6441f3-2166-47ef-9d4a-99ddd97aaa2d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:11.653809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 566/1152] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:11 2022] GET /identity/v3/projects/9b5cc7402cba44f0b26e551247d0e961 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:11.660003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:11.755096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef81ff6-b3d1-408d-909e-974359f6f344 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:11.774102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:11.788052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:11.788911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:11.789198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:11.789473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] PATH_INFO: `/v3/projects/9b5cc7402cba44f0b26e551247d0e961/users/75af96d2691041249e24e2fa646e4c43/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:11.791427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b5cc7402cba44f0b26e551247d0e961, user_id=75af96d2691041249e24e2fa646e4c43, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:11.792845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acfde7-8c1b-4532-9b0d-e22d2ebfd7b1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:11.861467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 587/1153] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:37:11 2022] PUT /identity/v3/projects/9b5cc7402cba44f0b26e551247d0e961/users/75af96d2691041249e24e2fa646e4c43/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:37:13.996869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bdf1-a8df-45f5-a7a5-386f6bf53392 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:13.997394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bdf1-a8df-45f5-a7a5-386f6bf53392 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:13.997484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bdf1-a8df-45f5-a7a5-386f6bf53392 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:13.998851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 567/1154] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:14.003970 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.004334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.004530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.021407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:14.022576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:14.086104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 588/1155] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:14.108952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bdf1-a8df-45f5-a7a5-386f6bf53392 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:14.167438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:14.209044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bdf1-a8df-45f5-a7a5-386f6bf53392 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:14.240767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:14.241542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.241829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.242105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.252534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:14.260049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:14.261144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:37:14.262192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 568/1156] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:14.267667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:14.362835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085956d-9e3e-49a2-95d5-24e5cf423552 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:14.372905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:14.396158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:14.396814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.397102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.397378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.398121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:14.398433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:14.399835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:14.407989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 589/1157] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:14.413773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:14.510868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a668e9-c8c3-44be-b97d-de55223ceb9f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:14.526646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:14.542400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:14.543288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.543572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.543845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] PATH_INFO: `/v3/users/75af96d2691041249e24e2fa646e4c43` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.545119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] RBAC: Authorizing `identity:get_user(user_id=75af96d2691041249e24e2fa646e4c43)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:14.546545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:14.548153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 569/1158] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/users/75af96d2691041249e24e2fa646e4c43 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:14.554183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:14.649633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784447f-7e1d-487e-8f1c-e6783f49fae1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:14.681404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:14.682033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.682319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.682599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.722547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:14.746872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:14.748315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:14.842185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 590/1159] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:37:14.848526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:14.942988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfec48-c8c7-47fc-8284-d706b2d60775 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:14.974941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:14.975778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:14.976060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:14.976333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] PATH_INFO: `/v3/projects/9b5cc7402cba44f0b26e551247d0e961` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:14.977495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] RBAC: Authorizing `identity:get_project(project_id=9b5cc7402cba44f0b26e551247d0e961)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:14.978917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0451a5-e023-4635-865a-7e6a17f3b4ec None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:14.980550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 570/1160] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/projects/9b5cc7402cba44f0b26e551247d0e961 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:14.986481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:15.052984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:15.081315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f424-c209-4ed3-b982-5c36b08d0846 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:15.113514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:15.114095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:15.114375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:15.114669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:15.115487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=9b5cc7402cba44f0b26e551247d0e961, user.id=75af96d2691041249e24e2fa646e4c43) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:15.115799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:15.117203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d34c1-744b-409f-b5ed-5b8f28431f9e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:15.122893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 591/1161] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:37:14 2022] GET /identity/v3/role_assignments?user.id=75af96d2691041249e24e2fa646e4c43&scope.project.id=9b5cc7402cba44f0b26e551247d0e961&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:17.318398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385828-8e2e-444a-95df-00af12305d3f None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:17.319055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385828-8e2e-444a-95df-00af12305d3f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:17.319055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385828-8e2e-444a-95df-00af12305d3f None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:17.320354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 571/1162] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:17.325593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:17.326041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:17.326152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:17.342799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:17.343958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:17.407093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 592/1163] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:17.430468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385828-8e2e-444a-95df-00af12305d3f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:17.484977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:17.529881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385828-8e2e-444a-95df-00af12305d3f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:17.562450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:17.563109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:17.563394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:17.563675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:17.628091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:17.629470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:17.680436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:17.724359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 572/1164] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:37:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:37:17.730053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:17.791514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:17.823951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232bb9e-c5cb-4453-8da8-05518fe42b28 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:17.856865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:17.857644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:17.857928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:17.858202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:17.868892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:17.870327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-ba06e370-f730-4dce-82e9-15bcb5134f26 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:17.884642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:17.885896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:17.887147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 593/1165] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:37:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:17.891730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:17.988413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae5090-d0be-4342-9f5f-b70109995831 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:18.021187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:18.021829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:18.022111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:18.022411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:18.023156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:18.023465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:18.024802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c76d1-04c0-433e-ac05-dbf58f15a176 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:18.031606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 573/1166] 10.222.0.74 () {56 vars in 1115 bytes} [Wed May 25 14:37:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:20.163567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4461a-dcb1-4816-8ec0-8031d7266124 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:20.164120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4461a-dcb1-4816-8ec0-8031d7266124 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:20.164120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4461a-dcb1-4816-8ec0-8031d7266124 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:20.165468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 594/1167] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:20.170598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035ded-a2e4-46af-ab41-512e082133db None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:20.171019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035ded-a2e4-46af-ab41-512e082133db None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:20.171184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035ded-a2e4-46af-ab41-512e082133db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:20.188110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-29035ded-a2e4-46af-ab41-512e082133db None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:20.189289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-29035ded-a2e4-46af-ab41-512e082133db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:20.201973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:20.254440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 574/1168] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:20.276615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4461a-dcb1-4816-8ec0-8031d7266124 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:20.377320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4461a-dcb1-4816-8ec0-8031d7266124 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:20.401401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:20.410223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:20.410910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:20.411198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:20.411477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:20.412216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:20.413649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53ba2-2843-4db2-932e-eaf7c4b3f7d6 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:20.467162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 595/1169] 10.222.0.74 () {60 vars in 1138 bytes} [Wed May 25 14:37:20 2022] POST /identity/v3/domains => generated 267 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:37:22.645205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef550fa-d50a-4fee-b13b-f74d9a8c30c8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:22.645718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef550fa-d50a-4fee-b13b-f74d9a8c30c8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:22.645718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef550fa-d50a-4fee-b13b-f74d9a8c30c8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:22.647106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 575/1170] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:22.652175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:22.652613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:22.652744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:22.669717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:22.670946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:22.735407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 596/1171] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:22.758508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef550fa-d50a-4fee-b13b-f74d9a8c30c8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:22.855977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef550fa-d50a-4fee-b13b-f74d9a8c30c8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:22.887693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:22.888531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:22.888813 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:22.889089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] PATH_INFO: `/v3/domains/1b5f253e1d434dc0af5bad05cb0edde6` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:22.899748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b5f253e1d434dc0af5bad05cb0edde6)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:22.901119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9467-b58a-40e3-9b18-ff7fe4f36474 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:22.902737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 576/1172] 10.222.0.74 () {56 vars in 1183 bytes} [Wed May 25 14:37:22 2022] GET /identity/v3/domains/1b5f253e1d434dc0af5bad05cb0edde6 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:22.909083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:23.003911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693808-bf5b-4bc8-94bf-53acd2b2a9a8 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:23.019946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:23.037165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:23.038780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:23.038780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:23.038780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:23.039194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:23.040629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1d913-343c-4a8f-84c9-b58a4144b8ee None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:23.101919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 597/1173] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:37:22 2022] POST /identity/v3/projects => generated 361 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:37:25.267584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70fd3-28f1-4cbe-b8d9-ab5d49ef9343 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:25.268135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70fd3-28f1-4cbe-b8d9-ab5d49ef9343 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:25.268135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70fd3-28f1-4cbe-b8d9-ab5d49ef9343 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:25.269502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 577/1174] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:25.274656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:25.275071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:25.275214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:25.291525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:25.292692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:25.357261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 598/1175] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:25.382174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa70fd3-28f1-4cbe-b8d9-ab5d49ef9343 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:25.393277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:25.480845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa70fd3-28f1-4cbe-b8d9-ab5d49ef9343 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:25.513241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:25.514084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:25.514368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:25.514668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] PATH_INFO: `/v3/domains/1b5f253e1d434dc0af5bad05cb0edde6` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:25.515843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b5f253e1d434dc0af5bad05cb0edde6)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:25.517287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77c1d3-d198-4229-8daa-d83359f5430a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:25.518953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 578/1176] 10.222.0.74 () {56 vars in 1183 bytes} [Wed May 25 14:37:25 2022] GET /identity/v3/domains/1b5f253e1d434dc0af5bad05cb0edde6 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:25.524518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:25.616761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:25.620457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1fc40-ec67-46cc-b6ee-0ba9c53275ab None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:25.652613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:25.653266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:25.653551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:25.653830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:25.654546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:25.655997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ab03b-ff32-41aa-8758-b324e7e077cf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:25.727521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 599/1177] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:37:25 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:37:27.945563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eac5e-66c0-4e09-a106-c63632670a52 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:27.946120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eac5e-66c0-4e09-a106-c63632670a52 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:27.946120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eac5e-66c0-4e09-a106-c63632670a52 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:27.947686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 579/1178] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:27.952861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:27.953259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:27.955164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:27.970680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:27.971852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:28.035353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 600/1179] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:28.059612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eac5e-66c0-4e09-a106-c63632670a52 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.135173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:28.158422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eac5e-66c0-4e09-a106-c63632670a52 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:28.190727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:28.191500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:28.191787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:28.192068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:28.201817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:28.210131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:28.211118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:37:28.212156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 580/1180] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:28.217108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.297635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:28.313001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fc6d6-5a1e-4243-8ff5-8b037d93f5b8 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:28.347364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23213-e453-4829-babe-384951ddb808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:28.348027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23213-e453-4829-babe-384951ddb808 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:28.348314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23213-e453-4829-babe-384951ddb808 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:28.348595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23213-e453-4829-babe-384951ddb808 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:28.349344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23213-e453-4829-babe-384951ddb808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:28.349664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23213-e453-4829-babe-384951ddb808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:28.351522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23213-e453-4829-babe-384951ddb808 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:28.359761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 601/1181] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:28.366056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.462933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b48aa-f914-4f41-9b2c-67de1f7c312c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:28.495495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:28.496345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:28.496627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:28.496900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] PATH_INFO: `/v3/users/0bb6073cc73d4ea0acd2708c186b2630` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:28.499524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:28.509487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] RBAC: Authorizing `identity:get_user(user_id=0bb6073cc73d4ea0acd2708c186b2630)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:28.510953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:28.512567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 581/1182] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/users/0bb6073cc73d4ea0acd2708c186b2630 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:28.518235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23213-e453-4829-babe-384951ddb808 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.615118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23213-e453-4829-babe-384951ddb808 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:28.646731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:28.647370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:28.647659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:28.647941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:28.663726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:28.712926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:28.714418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:28.807929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 602/1183] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:37:28.813562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.909985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93518d83-5859-4812-b475-8079dcad9d19 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:28.942311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:28.943178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:28.943465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:28.943751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] PATH_INFO: `/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:28.944926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] RBAC: Authorizing `identity:get_project(project_id=0e5d8a7bb8a34818b9b03c5c8aebbf67)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:28.946335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448ddc6-c610-48e4-99d0-226486af5373 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:28.948028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 582/1184] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:28.953824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:28.998875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:29.049531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbeb9e-7db9-437c-a618-5b70aeceb197 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:29.082232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:29.083147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:29.083435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:29.083713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:29.084535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=0e5d8a7bb8a34818b9b03c5c8aebbf67, user.id=0bb6073cc73d4ea0acd2708c186b2630) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:29.084853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:29.086285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391527-6851-4cde-92a7-c0cd6a2f574d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:29.091563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 603/1185] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:37:28 2022] GET /identity/v3/role_assignments?user.id=0bb6073cc73d4ea0acd2708c186b2630&scope.project.id=0e5d8a7bb8a34818b9b03c5c8aebbf67&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:31.233938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0038c-1e97-419f-b63c-9b025c96d50a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.234493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0038c-1e97-419f-b63c-9b025c96d50a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.234493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0038c-1e97-419f-b63c-9b025c96d50a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.236401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 583/1186] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:31.241568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.241966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.242139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.259219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:31.260409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:31.325461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 604/1187] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:31.348053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0038c-1e97-419f-b63c-9b025c96d50a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:31.407390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:31.445612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0038c-1e97-419f-b63c-9b025c96d50a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:31.478297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:31.479097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.479383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.479664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.489862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:31.497613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:31.498740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:37:31.499989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 584/1188] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:31.505226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:31.544853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:31.600226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafe231-4514-4f85-af8a-93a4abed8eba None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:31.632673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:31.633317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.633597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.633870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.634601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:31.634963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:31.636318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:31.644819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 605/1189] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:31.650602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:31.746856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90f78-3fa8-47c7-be0c-fc3e7d94ea8c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:31.768401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:31.779131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:31.779980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.780259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.781884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] PATH_INFO: `/v3/users/0bb6073cc73d4ea0acd2708c186b2630` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.781884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] RBAC: Authorizing `identity:get_user(user_id=0bb6073cc73d4ea0acd2708c186b2630)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:31.783122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:31.784698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 585/1190] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:31 2022] GET /identity/v3/users/0bb6073cc73d4ea0acd2708c186b2630 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:31.790257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:31.886419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f1872-1b7d-489a-944f-b304e51c1a39 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:31.887936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:31.919360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:31.920223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:31.920510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:31.920785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] PATH_INFO: `/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:31.921961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] RBAC: Authorizing `identity:get_project(project_id=0e5d8a7bb8a34818b9b03c5c8aebbf67)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:31.923963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaec52-259a-497c-81bb-54f5de2f2d0d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:31.925610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 606/1191] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:31 2022] GET /identity/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:31.931339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:32.029162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c1cf-6f5d-42e9-8403-b0b54378c02f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:32.061393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:32.062256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:32.062538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:32.062844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] PATH_INFO: `/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67/users/0bb6073cc73d4ea0acd2708c186b2630/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:32.065365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e5d8a7bb8a34818b9b03c5c8aebbf67, user_id=0bb6073cc73d4ea0acd2708c186b2630, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:32.066820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505848d-0ee7-4cd9-b5d8-ab26dad333c0 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:32.134102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 586/1192] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:37:31 2022] PUT /identity/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67/users/0bb6073cc73d4ea0acd2708c186b2630/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:37:34.292047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df53be-be1b-481f-9635-deac4cdbafeb None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.292572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df53be-be1b-481f-9635-deac4cdbafeb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.292572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df53be-be1b-481f-9635-deac4cdbafeb None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:34.293941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 607/1193] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:37:34.299143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.299577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.299707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:34.316239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:37:34.317413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:37:34.362977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:34.383302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 587/1194] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:37:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:37:34.407083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df53be-be1b-481f-9635-deac4cdbafeb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:34.505663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df53be-be1b-481f-9635-deac4cdbafeb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:34.537994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97052-0497-4286-b45b-e04231762285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:34.538799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97052-0497-4286-b45b-e04231762285 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.539080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97052-0497-4286-b45b-e04231762285 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.539355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97052-0497-4286-b45b-e04231762285 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:34.549131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a97052-0497-4286-b45b-e04231762285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-73a97052-0497-4286-b45b-e04231762285 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:37:34.556630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:37:34.557570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:37:34.558439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 608/1195] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:37:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:37:34.563574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:34.659047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28af7b-49a2-491e-8b54-f072e9243616 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:34.691485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:34.692128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.692410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.692686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:34.693418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:34.693727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:34.695115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:34.702883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 588/1196] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:34.709213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97052-0497-4286-b45b-e04231762285 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:34.738919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:34.804169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97052-0497-4286-b45b-e04231762285 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:34.836974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:34.837821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.838107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.838388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] PATH_INFO: `/v3/users/0bb6073cc73d4ea0acd2708c186b2630` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:34.839598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] RBAC: Authorizing `identity:get_user(user_id=0bb6073cc73d4ea0acd2708c186b2630)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:34.841033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:34.842625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 609/1197] 10.222.0.74 () {56 vars in 1177 bytes} [Wed May 25 14:37:34 2022] GET /identity/v3/users/0bb6073cc73d4ea0acd2708c186b2630 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:34.848127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:34.890077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:34.944694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153417c-d43e-40e8-a140-fbf17a70122e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:34.976776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:34.977409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:34.977693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:34.977968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:35.042383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:35.043847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:35.084177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:35.139179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 589/1198] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:37:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:37:35.144974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:35.237654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:35.241323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0f12-2563-4a95-81d0-d5b75ad65faf None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:35.272955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:35.273805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:35.274088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:35.274366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] PATH_INFO: `/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:35.275566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] RBAC: Authorizing `identity:get_project(project_id=0e5d8a7bb8a34818b9b03c5c8aebbf67)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:35.276999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783edfa-a7a7-4378-8a29-ad7aafc0c88f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:35.278823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 610/1199] 10.222.0.74 () {56 vars in 1186 bytes} [Wed May 25 14:37:35 2022] GET /identity/v3/projects/0e5d8a7bb8a34818b9b03c5c8aebbf67 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:35.285391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:37:35.381958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb5f1f-ff97-451b-8e2b-5ca17705cc0a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:37:35.414990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:37:35.415632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:37:35.415918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:37:35.416197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:37:35.417023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=0e5d8a7bb8a34818b9b03c5c8aebbf67, user.id=0bb6073cc73d4ea0acd2708c186b2630) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:37:35.417342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:37:35.418787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b3d2e-fe82-4f79-bf63-ef481d16b14a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:37:35.425069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 590/1200] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:37:35 2022] GET /identity/v3/role_assignments?user.id=0bb6073cc73d4ea0acd2708c186b2630&scope.project.id=0e5d8a7bb8a34818b9b03c5c8aebbf67&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:37:45.289029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=17 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:37:45.421867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:52.848678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-8f33-4ffb-a704-bc587236cb0c None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:52.849276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-8f33-4ffb-a704-bc587236cb0c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:52.849276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-8f33-4ffb-a704-bc587236cb0c None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:52.850653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 611/1201] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:52.855985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:52.856433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:52.856575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:52.873150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:52.876118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:52.939085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 591/1202] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:38:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:38:52.964486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37c22-8f33-4ffb-a704-bc587236cb0c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:53.065790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37c22-8f33-4ffb-a704-bc587236cb0c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:53.100420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:53.101149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:53.101434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:53.101715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:53.111581 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:53.113032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:38:53.130618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:53.131705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:53.133209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:38:53.133209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:38:53.133209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:53.133209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:38:53.133209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 612/1203] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:38:53.137784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:53.234603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fc906-71a8-4a84-a5f0-2dd4da4cc884 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:53.263168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:53.268804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:53.269457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:53.269739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:53.270652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:53.271406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:53.271716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:53.273091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045951-294e-4d5d-b862-71aad2f04f1d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:53.283992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 592/1204] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:53.289729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:53.342305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:53.386529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888890-2ea2-4b58-8a0d-9425e6987b9d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:53.419774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:53.420465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:53.420749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:53.421029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:53.421730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:53.423184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45519-75bc-45d0-955b-3b510ae0cb6b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:53.516678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 613/1205] 10.222.0.74 () {60 vars in 1132 bytes} [Wed May 25 14:38:53 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:38:55.716587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de916f-58d6-477d-83aa-b94f5afb35aa None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:55.717121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de916f-58d6-477d-83aa-b94f5afb35aa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:55.717225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de916f-58d6-477d-83aa-b94f5afb35aa None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:55.718601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 593/1206] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:55.724828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:55.724828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:55.725282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:55.741895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:55.743141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:55.807208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 614/1207] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:38:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:38:55.829695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de916f-58d6-477d-83aa-b94f5afb35aa None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:55.929300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de916f-58d6-477d-83aa-b94f5afb35aa None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:55.962453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:55.963263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:55.963547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:55.963826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:55.974146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:38:55.981926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:55.989061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:38:55.989918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 594/1208] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:38:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:38:55.990670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.028148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:56.086961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597cfa-975b-4bd4-b6b7-6b2354bac944 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.119200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.119867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.120151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.120430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.121168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:56.121479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.122871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:56.131088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 615/1209] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:38:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:56.136793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.185465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:56.234384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce311e-6763-47c6-a1de-b3d4139ec13c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.267279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.268060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.268348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.268628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.279611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.281056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:38:56.298188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:56.299177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:56.300086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:38:56.300086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:38:56.300086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:56.300086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:38:56.302808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 595/1210] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:38:56.305127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.402396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:56.405083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc63de-69d4-41fc-b559-a80eb00c4df9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.437074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.437719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.438041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.438332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.439071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:56.439376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.440735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:56.452405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 616/1211] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:56.458836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.557131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d114-68df-412c-85f6-809122eb8b7b None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.558540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:56.590388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.591203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.591487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.591763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.599227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.600650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:38:56.615089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:56.616200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:38:56.617278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 596/1212] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:38:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:38:56.622043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.719668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c143f-b8c4-49b7-8e09-79256a0982b8 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.753994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.755274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.755466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.755740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.756565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:56.756876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.758316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:56.776607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 617/1213] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:38:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:56.782385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:56.882475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38c-2e5f-49a9-b04c-8c3aca604d82 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:56.916506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:56.917138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:56.917421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:56.917697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:56.946304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:56.983453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:56.984898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.080404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 597/1214] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:38:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:38:57.086514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:57.105828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:57.186385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181db508-ccdf-40cc-8830-ec2bb41c181d None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:57.219557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:57.220334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:57.220615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:57.220890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:57.230958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:57.232385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:38:57.250465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:57.251519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:57.256878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:38:57.256878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:38:57.256878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:38:57.256878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:38:57.256878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 618/1215] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:38:57.258059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:57.318700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:57.357811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd84e-bfa8-458c-b954-54f6cbd9ed06 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:57.390108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:57.390786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:57.391069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:57.391342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:57.392054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:57.392361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:57.393711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.405709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 598/1216] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:57.411377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:57.479175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:57.509832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73111817-72c4-4f0a-8b71-b3ec8faddf37 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:57.542825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:57.543610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:57.543910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:57.544413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:57.552448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:57.554038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:38:57.567545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:38:57.569155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:38:57.570832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 619/1217] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:38:57.576576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:57.677290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc6a6-2875-4a25-82a1-977312a5b9fc None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:57.699086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:57.710265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:57.710938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:57.711226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:57.711502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:57.712273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:57.712582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:57.713984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2235-8417-45d0-b23e-7eed393080f9 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.725564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 599/1218] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:57.742834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:57.829984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39373-0b9e-42fe-beef-ae2bcb527907 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:57.858413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:57.864707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:57.865353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:57.865637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:57.865913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:57.866789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=dcd162deab164e70b629f8c88495ccb3) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:38:57.867107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:57.868536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51c6cd-445d-4cd2-bbe0-83a52b0dcf6b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:57.873875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 620/1219] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:38:57 2022] GET /identity/v3/role_assignments?user.id=dcd162deab164e70b629f8c88495ccb3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:39:00.027634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934aa5-c4c1-429b-9c1e-ea71e7115dfd None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.028212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934aa5-c4c1-429b-9c1e-ea71e7115dfd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.028212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934aa5-c4c1-429b-9c1e-ea71e7115dfd None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.029629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 600/1220] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:00.034725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.035110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.035793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.052817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:00.056223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:00.119805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 621/1221] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:00.142552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934aa5-c4c1-429b-9c1e-ea71e7115dfd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.243856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934aa5-c4c1-429b-9c1e-ea71e7115dfd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:00.276983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:00.277811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.278087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.278429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.289529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:39:00.297184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:00.298360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:00.299420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 601/1222] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:00.304383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.401353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39af4f-e6d3-463b-8279-45952e7333dc None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:00.424342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:00.434730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:00.435392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.435677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.436679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.436679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] RBAC: Adding query filter params (name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:00.437040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:00.438409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:00.446016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 622/1223] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:00.451916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.496766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:00.551679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf49bd2-86f1-423a-aa19-0ec686694f97 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:00.584570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:00.585352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.585659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.585919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.596134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:00.597569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:00.615472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:00.616513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:00.617479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:00.617479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:00.617479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:00.617479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:00.617479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 602/1224] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:00.622483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.720974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6c28-39a3-4821-8e4c-99397cf8cea7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:00.754418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:00.755109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.755394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.755673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.756389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:00.756705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:00.758075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:00.769364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 623/1225] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:00.775102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.867731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:00.874366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b60453-7b1d-436d-8ba2-bbb152365b1f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:00.907188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:00.907979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:00.908264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:00.908539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:00.916458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:00.917892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:39:00.932446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:00.933576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:00.934845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 603/1226] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:00.939359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:00.986900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:01.037369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1879b5-a6ed-4f4d-878d-11023fee5b0b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.069145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.069830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.070112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.070388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.071241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:01.071560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.072984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:01.090826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 624/1227] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:01.096382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:01.193945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce52399-cf14-440f-9fdc-7174d11c2d6a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.227987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.228761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.229045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.229367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.239152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.240589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:01.259083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:01.260146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:01.261121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:01.261121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:01.261121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:01.261121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:01.261121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 604/1228] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:01.266243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:01.368175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7613c6-4701-4b33-9b70-3c8cc9b1bf22 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.370429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:01.401337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.401995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.402283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.402565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.403312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:01.403628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.405004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:01.415969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 625/1229] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:01.421670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:01.430371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:01.517279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daa960-951b-45fe-9d79-2f1d36ee67d8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.551381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.552167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.552453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.552732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.559373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.560818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:39:01.574621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:01.575665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:01.576971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 605/1230] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:01.582080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:01.684699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2afe0-723b-4354-b5d7-85cd3d7ece4f None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.718327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.719015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.719296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.719570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.720339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:01.720683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.722111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5b78e-71dd-472c-9237-91a55bef9855 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:01.734436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 626/1231] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:01.740518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:01.806766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:01.840239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9f223-5acc-443e-8fa7-c0b397dfa96d None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:01.872456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:01.873324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:01.873633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:01.873907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/dcd162deab164e70b629f8c88495ccb3/roles/ba9ceb64deea49a0ae0ace2ba22b002d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:01.888121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=dcd162deab164e70b629f8c88495ccb3, role_id=ba9ceb64deea49a0ae0ace2ba22b002d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:01.889562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4033a9-a55e-4ff5-94b5-970654abf24c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:02.006891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 606/1232] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:39:01 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/dcd162deab164e70b629f8c88495ccb3/roles/ba9ceb64deea49a0ae0ace2ba22b002d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:39:04.172313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597198-dcba-4a6e-96f4-5fafe6754679 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.172870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597198-dcba-4a6e-96f4-5fafe6754679 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.172966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c597198-dcba-4a6e-96f4-5fafe6754679 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.174252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 627/1233] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:04.179550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.179957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.180112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.196585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:04.197771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:04.263700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 607/1234] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:04.287514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c597198-dcba-4a6e-96f4-5fafe6754679 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:04.312348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:04.387860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c597198-dcba-4a6e-96f4-5fafe6754679 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:04.421508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:04.422299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.422668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.422887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.433111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:39:04.440923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:04.442526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:04.448459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 628/1235] 10.222.0.74 () {56 vars in 1102 bytes} [Wed May 25 14:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:04.449161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:04.495446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:04.546170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96e1c6-b7fe-45db-a8e3-e2ea650bae2c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:04.578671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:04.579750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.580207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.580849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.581907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] RBAC: Adding query filter params (name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:04.582778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:04.586142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:04.592406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 608/1236] 10.222.0.74 () {56 vars in 1103 bytes} [Wed May 25 14:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:39:04.598025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:04.681349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:04.696772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135a99-8e1a-4d13-8f6c-783ff063a2b2 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:04.730611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:04.731418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.731706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.731990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.742850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:04.744787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:04.762345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:04.763485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:04.768671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:04.768671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:04.768671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:04.768671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:04.768671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 629/1237] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:04.769616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:04.868241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103250b-14bb-4d8d-9776-d23edba99474 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:04.877022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:04.901215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:04.901865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:04.902152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:04.902432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:04.903189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:04.903532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:04.904893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:04.916434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 609/1238] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:04.922316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.020130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a502eb-e068-4ae0-a923-f8fecf3afff3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.054695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.055496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.055781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.056057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:05.058870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:05.063511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:05.064950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:39:05.079415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:05.080530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:05.081533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 630/1239] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:05.087020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.184950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142831bb-b2a5-4f0a-9226-0ddf2ab9e8ac None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.217836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.218503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.218824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.219107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:05.219939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:05.220253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:05.221679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:05.240239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 610/1240] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:39:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:05.245994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.344578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fcef7-dada-4a58-91d6-3224f666f1f9 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.377345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.377988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.378273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.378550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:05.438362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:05.445372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:05.446853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:05.539493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 631/1241] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:39:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:39:05.545363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.572537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:05.643386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d95365-04d0-44c1-b6a8-5a627acea12f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.677556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.678335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.678614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.678919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:05.688953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:05.690402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:05.708483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:05.709539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:05.710495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:05.710495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:05.710495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:05.710495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:05.710495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 611/1242] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:05.715803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.808413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:05.814085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9ce7c-1e3c-46fc-8979-39e2a30e8b99 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.846998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.847668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.848463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.848742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:05.849491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:05.849804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:05.851204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:05.862395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 632/1243] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:05.868309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:05.944021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:05.964320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cf76c-192d-481e-8113-bc235594df86 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:05.997479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:05.998261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:05.998545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:05.998850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:06.005308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:06.006761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:39:06.020204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:06.021162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:06.022063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:06.030015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 612/1244] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:06.030111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:06.126663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576e33c-034c-416d-8a5f-bd742a12316b None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:06.160236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:06.160892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:06.161179 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:06.161457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:06.162236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:06.162551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:06.164014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee58c9-b7fd-4fd5-abbd-f970ed88c52b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:06.176316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 633/1245] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:06.182370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:06.280379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d805-9be2-4546-8c47-a8e02d71fa56 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:06.313124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:06.313781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:06.314072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:06.314353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:06.315229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] RBAC: Adding query filter params (role.id=ba9ceb64deea49a0ae0ace2ba22b002d, scope.project.id=c07159d21300478e88a07006614fdded, user.id=dcd162deab164e70b629f8c88495ccb3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:06.315550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:06.317553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d099d-f833-4c3c-9b14-ef3594624f13 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:06.323760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 613/1246] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:39:06 2022] GET /identity/v3/role_assignments?user.id=dcd162deab164e70b629f8c88495ccb3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=ba9ceb64deea49a0ae0ace2ba22b002d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:08.549993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600742d1-3461-4fec-96bb-abbf986a8aa9 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:08.550547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600742d1-3461-4fec-96bb-abbf986a8aa9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:08.550547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600742d1-3461-4fec-96bb-abbf986a8aa9 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:08.551938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 634/1247] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:08.557366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:08.557792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:08.557908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:08.562919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:08.574892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:08.576104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:08.641455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 614/1248] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:08.665648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600742d1-3461-4fec-96bb-abbf986a8aa9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:08.683320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:08.764767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600742d1-3461-4fec-96bb-abbf986a8aa9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:08.798054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:08.798855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:08.799139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:08.799412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:08.809213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:39:08.817351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:08.818448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:08.819403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 635/1249] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:39:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:08.824537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:08.921790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666eb87-44f5-4ec6-bd3f-5f9cd76e60cf None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:08.950178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:08.955097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:08.957424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:08.957424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:08.957424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:08.957424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:08.957836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:08.958979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:08.967129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 615/1250] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:39:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:08.973285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:09.048306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:09.070666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442bec-9816-44a3-a705-4c1c2bb501bd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:09.103871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:09.104646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:09.104929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:09.105249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:09.116325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:09.118001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:09.135109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:09.136132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:09.137360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:09.137360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:09.137360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:09.137360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:09.137360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 636/1251] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:09.141982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:09.242748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa93e2d-2d7a-4fde-9f15-b5ae91d4f1dd None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:09.275476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:09.276129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:09.276411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:09.276687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:09.277403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:09.277714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:09.279162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:09.290483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 616/1252] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:09.296027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:09.393721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161600-64df-46f5-bf89-d0d20aea2ae1 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:09.418172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:09.427547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:09.428343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:09.428627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:09.428903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:09.436326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:09.437757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:39:09.452145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:09.458770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:09.459996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 637/1253] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:09.460719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:09.514996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:09.556629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bae15-80d5-4291-a41f-58a0af3cab5e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:09.589976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:09.590630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:09.590938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:09.591213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:09.592035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:09.592344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:09.593759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:09.610895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 617/1254] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:39:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:09.616696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:09.717354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c1321-8556-4c66-a533-32f4ed62b530 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:09.749138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:09.749763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:09.750043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:09.750316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:09.804282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:09.816012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:09.818041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:09.914021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 638/1255] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:39:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:39:09.919885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:10.018360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cff2a5-2f4f-41af-9dfd-fe4e09926877 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:10.031285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:10.052039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:10.052820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:10.053106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:10.053391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:10.063164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:10.064551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:10.082763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:10.083866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:10.085059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:10.085059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:10.085059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:10.085059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:10.085059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 618/1256] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:10.089692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:10.173351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:10.185100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e0a04-4c54-49c2-9f38-b6045b3b6e51 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:10.217355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:10.218060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:10.218322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:10.218597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:10.219336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:10.219643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:10.220990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:10.231682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 639/1257] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:10.237289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:10.336320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0414c-4375-47bc-b798-888b86e60ae4 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:10.369784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:10.370579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:10.370898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:10.371178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:10.377820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:10.379837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:39:10.393065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:10.394122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:10.401624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 619/1258] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:10.402268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:10.497699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295b8fc-757c-4323-92a5-5fb3a4a7a41a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:10.530618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:10.531297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:10.531579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:10.531855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:10.532633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:10.532942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:10.534367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5d1af-6021-4941-917f-ad1c872645f9 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:10.545815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 640/1259] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:10.554196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:10.578070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:10.652401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035284f-9392-49ce-a1e4-5861e57a5ff2 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:10.686166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:10.686833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:10.687117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:10.687393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:10.688314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=dcd162deab164e70b629f8c88495ccb3) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:10.688678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:10.690142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ba50-64a0-4b6e-b46e-debda82ba85b None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:10.696030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 620/1260] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:39:10 2022] GET /identity/v3/role_assignments?user.id=dcd162deab164e70b629f8c88495ccb3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:39:12.880010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea683-9c06-47bd-a917-8994fe71d776 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:12.880563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea683-9c06-47bd-a917-8994fe71d776 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:12.880563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea683-9c06-47bd-a917-8994fe71d776 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:12.881899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 641/1261] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:12.887031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:12.887409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:12.887586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:12.904250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:12.905415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:12.971771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 621/1262] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:12.994320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea683-9c06-47bd-a917-8994fe71d776 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.005369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.092864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adea683-9c06-47bd-a917-8994fe71d776 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.125913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.126759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.127041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.127315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.137242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:39:13.145507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:13.146583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:13.152430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 642/1263] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:13.153273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.171512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.252591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6164de2-32ea-4595-b180-b812ac2a1200 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.288989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.289655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.289936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.290219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.290984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:13.291292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:13.292650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:13.300788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 622/1264] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:13.306617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.377290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.411710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ed47-015c-44c9-93d6-01427b7668e3 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.454943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.455811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.456104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.456402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.468684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:13.470159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:13.487427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:13.488396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:13.489307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:13.489307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:13.489307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:13.489307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:13.489307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 643/1265] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:13.495598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.575695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.595586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343989ba-d5e1-44d7-8083-bd7a4d4e4f66 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.627781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.628432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.628716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.629521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.630239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:13.630550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:13.631934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:13.643572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 623/1266] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:13.650118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.750163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb429f-72fa-4e9a-a415-c7444a8b2fa7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.770286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.783716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.784517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.784797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.785079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.792586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:13.794031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:39:13.808723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:13.809821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:13.810866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 644/1267] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:13.815739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:13.913003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a0a7-6f20-4a61-842e-514e6c001a11 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:13.946061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:13.946754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:13.947039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:13.947317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:13.948172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:13.948485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:13.949916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:13.959122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:13.966982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 624/1268] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:13.972481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:14.071075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143edfe7-bab4-48e2-b6ae-2fa42cdfc9f4 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:14.103915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:14.104687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:14.104968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:14.105248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:14.116419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:14.117869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:14.134672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:14.135691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:14.136776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:14.136776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:14.136776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:14.136776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:14.138789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 645/1269] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:14.141525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:14.241093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf91baa-d1bf-4011-b148-b5a6a43cffb8 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:14.273934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:14.274735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:14.275026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:14.275303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:14.276049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:14.276359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:14.277730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:14.289167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 625/1270] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:14.294831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:14.326797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:14.393552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994aac-8d54-4bc5-8c98-e61c75547080 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:14.428840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:14.429870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:14.430158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:14.430438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:14.437114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:14.438570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:39:14.452880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:14.455021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:14.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 646/1271] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:14.460665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:14.518613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:14.559469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0feb-dbc4-45c6-8984-02200349263f None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:14.592207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:14.592803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:14.593090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:14.593369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:14.594236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:14.594551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:14.596029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18a3e2-3fd5-4527-bed8-cb537c000d1c None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:14.608797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 626/1272] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:14.614861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:14.709878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:14.718272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de53d38-a7ee-48e1-9e05-93de27401061 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:14.751630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:14.752516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:14.752802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:14.753081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] PATH_INFO: `/v3/projects/c07159d21300478e88a07006614fdded/users/dcd162deab164e70b629f8c88495ccb3/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:14.755621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] RBAC: Authorizing `identity:create_grant(project_id=c07159d21300478e88a07006614fdded, user_id=dcd162deab164e70b629f8c88495ccb3, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:14.757065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d6c75-bf46-417c-8748-ef39f77fa33b None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:14.933789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 647/1273] 10.222.0.74 () {58 vars in 1439 bytes} [Wed May 25 14:39:14 2022] PUT /identity/v3/projects/c07159d21300478e88a07006614fdded/users/dcd162deab164e70b629f8c88495ccb3/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:39:17.182925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d144e-08de-4668-a15b-19559d16d5f7 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.183499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d144e-08de-4668-a15b-19559d16d5f7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.183619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d144e-08de-4668-a15b-19559d16d5f7 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.184914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 627/1274] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:17.190182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.190579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.190769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.207545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:17.208718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:17.276260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 648/1275] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:17.299974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d144e-08de-4668-a15b-19559d16d5f7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:17.401708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d144e-08de-4668-a15b-19559d16d5f7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:17.419534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:17.435965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:17.436685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.436970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.437246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.447666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:39:17.456060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:17.463099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:17.463950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 628/1276] 10.222.0.74 () {56 vars in 1096 bytes} [Wed May 25 14:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:17.464651 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:17.545196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:17.562307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62723fa-9ac8-4026-9281-6ee53f7599d4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:17.595044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:17.595779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.596064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.596339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.597076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:17.597384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:17.599348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:17.607037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 649/1277] 10.222.0.74 () {56 vars in 1099 bytes} [Wed May 25 14:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:17.612672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:17.712626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28386-741e-4d91-ac1b-3b175462dd9c None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:17.752584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca992b-711e-4653-be92-5db262354965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:17.753368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca992b-711e-4653-be92-5db262354965 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.753655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca992b-711e-4653-be92-5db262354965 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.754470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca992b-711e-4653-be92-5db262354965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.765040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca992b-711e-4653-be92-5db262354965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:17.766520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca992b-711e-4653-be92-5db262354965 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-7eca992b-711e-4653-be92-5db262354965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:17.783989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:17.785043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:17.786218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:17.786218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:17.786218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:17.786218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:17.786218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 629/1278] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:17.792561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:17.891311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152c90-6d3a-4a16-9576-d09a74355b4a None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:17.925159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:17.925932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:17.926219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:17.926548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:17.927303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:17.929229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:17.929229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:17.940601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 650/1279] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:17.946743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca992b-711e-4653-be92-5db262354965 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:17.977326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:18.044747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca992b-711e-4653-be92-5db262354965 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:18.079096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:18.079937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:18.080219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:18.080491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:18.088309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:18.089750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:39:18.104082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:18.108357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:18.109434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 630/1280] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:18.111267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:18.115057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:18.207859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d629fc-80d4-40ce-a1fc-458b20c5fa76 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:18.240531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:18.241120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:18.241401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:18.241676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:18.242488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:18.242830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:18.244250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:18.261153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 651/1281] 10.222.0.74 () {56 vars in 1143 bytes} [Wed May 25 14:39:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:18.267047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:18.357982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:18.366861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1f0a0-d7d1-477b-a6d0-369730f33550 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:18.399491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:18.400146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:18.400413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:18.400687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:18.465701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:18.467168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:18.552750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:18.562524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 631/1282] 10.222.0.74 () {58 vars in 1282 bytes} [Wed May 25 14:39:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:39:18.568909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:18.628716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:18.667375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab93e2-6777-4f62-a40e-910b60d800cd None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:18.700341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:18.701120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:18.701402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:18.701676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:18.713090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:18.714531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:39:18.731201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:18.732259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:18.733154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:39:18.733154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:39:18.733154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:39:18.733154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:18.733154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 652/1283] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:18.738081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:18.837067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a599-7f89-485a-ae5e-7932430d6047 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:18.869203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:18.869864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:18.870147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:18.870423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:18.871173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:18.871486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:18.872833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:18.884382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 632/1284] 10.222.0.74 () {56 vars in 1109 bytes} [Wed May 25 14:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:18.889999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:18.987151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b019-8d6d-44bc-997d-e866fbc23af7 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:19.000787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:19.020399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:19.021176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:19.021457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:19.021732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:19.028225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:19.029660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:39:19.043502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application with Lock( May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:19.044580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: ERROR keystone.server.flask.application May 25 14:39:19.045656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 653/1285] 10.222.0.74 () {56 vars in 1111 bytes} [Wed May 25 14:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:19.050478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:19.100061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:19.148303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e597-b778-4d8c-b57f-f78d043e497e None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:19.182013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:19.182696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:19.182985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:19.183265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:19.184050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:19.184362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:19.185777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685e780-8a55-409f-a4d2-2af643590e8d None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:19.199447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 633/1286] 10.222.0.74 () {56 vars in 1148 bytes} [Wed May 25 14:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:19.205685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:19.302207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f72a9-bb48-4795-af4c-2c65123db267 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:19.337695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] RBAC: Adding query filter params (role.id=d38d6cda16454360a4f48025ab4df728, scope.project.id=c07159d21300478e88a07006614fdded, user.id=dcd162deab164e70b629f8c88495ccb3) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:19.342622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000dbe2-88bd-4ee2-9847-bad7edbde0da None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:19.347293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 654/1287] 10.222.0.74 () {56 vars in 1374 bytes} [Wed May 25 14:39:19 2022] GET /identity/v3/role_assignments?user.id=dcd162deab164e70b629f8c88495ccb3&scope.project.id=c07159d21300478e88a07006614fdded&role.id=d38d6cda16454360a4f48025ab4df728 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:21.548782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a88fb-08aa-4f1e-b9c4-8ada31daa1e6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:21.549352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a88fb-08aa-4f1e-b9c4-8ada31daa1e6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:21.549352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a88fb-08aa-4f1e-b9c4-8ada31daa1e6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:21.550762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 634/1288] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:21.556079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:21.556514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:21.556661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:21.573642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:21.575469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:21.604233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:21.640041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 655/1289] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:21.662072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a88fb-08aa-4f1e-b9c4-8ada31daa1e6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:21.761165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a88fb-08aa-4f1e-b9c4-8ada31daa1e6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:21.771993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:21.793753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:21.794531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:21.794839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:21.795112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:21.795754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:21.797106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:21.808715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:21.809971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:21.816005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 635/1290] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:39:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:21.817660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:21.914017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9afbf7-79eb-4ce3-aff3-5699747acfa7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:21.947382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:21.954300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754afc12-6f2f-4d0a-a868-705dd6149604 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:21.955621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 656/1291] 10.222.0.74 () {56 vars in 1116 bytes} [Wed May 25 14:39:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:21.961742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:22.058660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18c2e4-2a8a-45a2-883e-413c46a31351 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:22.092017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:22.092670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:22.092957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:22.093238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] PATH_INFO: `/v3/services` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:22.093941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] RBAC: Adding query filter params (type=placement) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:22.094260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:22.095672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b28e81-125b-43e5-9fa6-e0c5602c47ab None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:22.100099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 636/1292] 10.222.0.74 () {56 vars in 1116 bytes} [Wed May 25 14:39:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:39:24.267405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21455-78fd-45b6-936f-465d20282b3e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:24.267957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21455-78fd-45b6-936f-465d20282b3e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:24.267957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21455-78fd-45b6-936f-465d20282b3e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:24.269316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 657/1293] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:24.274508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590af9-4627-418e-b6b0-3bc15dc47d31 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:24.275052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590af9-4627-418e-b6b0-3bc15dc47d31 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:24.275172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1590af9-4627-418e-b6b0-3bc15dc47d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:24.291457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b1590af9-4627-418e-b6b0-3bc15dc47d31 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:24.292632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b1590af9-4627-418e-b6b0-3bc15dc47d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:24.322405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:24.358928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 637/1294] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:24.382735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca21455-78fd-45b6-936f-465d20282b3e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:24.412824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:24.481142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca21455-78fd-45b6-936f-465d20282b3e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:24.515491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:24.516146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:24.516431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:24.516709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:24.517475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:24.518832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9705e-27db-4fb2-9433-f28d7f7f350d None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:24.600556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 658/1295] 10.222.0.74 () {60 vars in 1141 bytes} [Wed May 25 14:39:24 2022] POST /identity/v3/services => generated 246 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:39:26.775515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a974e-d8d6-4f99-9643-ad9cda85b0e1 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:26.776064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a974e-d8d6-4f99-9643-ad9cda85b0e1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:26.776179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a974e-d8d6-4f99-9643-ad9cda85b0e1 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:26.777509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 638/1296] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:26.782762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:26.783193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:26.783322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:26.800212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:26.801959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:26.871816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 659/1297] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:26.893820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a974e-d8d6-4f99-9643-ad9cda85b0e1 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:26.993226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a974e-d8d6-4f99-9643-ad9cda85b0e1 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:27.026694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:27.027416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:27.027696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:27.027971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] PATH_INFO: `/v3/services/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:27.028611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:27.030088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:27.042321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:27.043495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:27.044538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 639/1298] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:39:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:27.049760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:27.068459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:27.148786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52428377-16d6-4083-846a-ca5db7b7839c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:27.181090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:27.181747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:27.182034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:27.182315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:27.183051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:27.189513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:27.189513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:27.189687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 660/1299] 10.222.0.74 () {56 vars in 1116 bytes} [Wed May 25 14:39:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:27.196215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:27.227591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:27.293755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50be9d-a4ec-461d-8bf4-8e7ce808d42a None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:27.326783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:27.327435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:27.327742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:27.328038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:27.328983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] RBAC: Adding query filter params (interface=public, service_id=607b6bb8b05248d0bbafafd825daa0d5, region_id=RegionOne) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:27.329343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:27.331400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c583d25-f05e-4289-b438-ec848adb21c2 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:27.336825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 640/1300] 10.222.0.74 () {56 vars in 1251 bytes} [Wed May 25 14:39:27 2022] GET /identity/v3/endpoints?service_id=607b6bb8b05248d0bbafafd825daa0d5&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:27.342365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:27.422218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:27.439848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e23f0-3258-4b78-9ec4-95e546640bc0 None admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:27.472437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:27.473138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:27.473404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:27.473685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:27.474361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:27.475781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d403d4-5e09-4747-aede-3fa11ea3b2c2 None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:27.483238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 661/1301] 10.222.0.74 () {56 vars in 1087 bytes} [Wed May 25 14:39:27 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:39:29.663552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe522e09-8c32-4044-aeff-fcdb8c3d309f None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:29.664072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe522e09-8c32-4044-aeff-fcdb8c3d309f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:29.664072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe522e09-8c32-4044-aeff-fcdb8c3d309f None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:29.665413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 641/1302] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:39:29.670820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:29.671252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:29.671373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:29.688607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:29.689833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:29.755254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 662/1303] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:39:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:39:29.777699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe522e09-8c32-4044-aeff-fcdb8c3d309f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:29.869937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:29.876295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe522e09-8c32-4044-aeff-fcdb8c3d309f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:29.908472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:29.909286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:29.909756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:29.910824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:29.911856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:29.913242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:39:29.925236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:39:29.927108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application with Lock( May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self._enter() May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application return self.creator() May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application created_value = creator( May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: ERROR keystone.server.flask.application May 25 14:39:29.928166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 642/1304] 10.222.0.74 () {56 vars in 1117 bytes} [Wed May 25 14:39:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:39:29.933189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:30.007191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:30.028787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2860197-dccd-4b43-8f59-249c4d9c0e5a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:30.061784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:30.062433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:30.062757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:30.063039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] PATH_INFO: `/v3/services` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:30.063733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] RBAC: Adding query filter params (name=placement) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:39:30.064047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:30.065410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de6a83-746a-4934-ab36-6b8646afe57e None admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:30.070423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 663/1305] 10.222.0.74 () {56 vars in 1116 bytes} [Wed May 25 14:39:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:30.076160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:30.174349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ad54b-5d47-4025-9597-2a09feaef259 None admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:30.207672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4903e9-2753-437d-9868-542569460119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:30.208260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4903e9-2753-437d-9868-542569460119 None admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:30.208544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4903e9-2753-437d-9868-542569460119 None admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:30.208821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4903e9-2753-437d-9868-542569460119 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:30.209555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4903e9-2753-437d-9868-542569460119 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:30.210948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4903e9-2753-437d-9868-542569460119 None admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:30.314928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 643/1306] 10.222.0.74 () {60 vars in 1144 bytes} [Wed May 25 14:39:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:39:40.100038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=30 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:39:40.310202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:39.870266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cb442-2251-42c6-8e4a-b9c0b059ea67 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:39.870783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cb442-2251-42c6-8e4a-b9c0b059ea67 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:39.870870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cb442-2251-42c6-8e4a-b9c0b059ea67 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:39.878293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0153-1fd6-41ad-b5bd-1e75ed49220b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:39.878790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0153-1fd6-41ad-b5bd-1e75ed49220b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:39.878869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd0153-1fd6-41ad-b5bd-1e75ed49220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:39.881193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 664/1307] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:39 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:39.970666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fcdd0153-1fd6-41ad-b5bd-1e75ed49220b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:39.971912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fcdd0153-1fd6-41ad-b5bd-1e75ed49220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:40.103096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 644/1308] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:45:40.136847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496769be-1070-4fd3-a009-df0696353c33 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:40.137525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496769be-1070-4fd3-a009-df0696353c33 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:40.138064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496769be-1070-4fd3-a009-df0696353c33 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:40.139857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 665/1309] 10.222.0.74 () {54 vars in 964 bytes} [Wed May 25 14:45:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:40.146888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad97c5-a956-4355-a91a-288cf83976ad None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:40.147548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad97c5-a956-4355-a91a-288cf83976ad None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:40.147975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad97c5-a956-4355-a91a-288cf83976ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:40.233274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f0ad97c5-a956-4355-a91a-288cf83976ad None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:40.234932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f0ad97c5-a956-4355-a91a-288cf83976ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:40.257693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:40.325201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 645/1310] 10.222.0.74 () {58 vars in 1061 bytes} [Wed May 25 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:40.337166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496769be-1070-4fd3-a009-df0696353c33 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:40.339008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-496769be-1070-4fd3-a009-df0696353c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:40.449590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496769be-1070-4fd3-a009-df0696353c33 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:40.525219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:40.525951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:40.526334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:40.526743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:40.528493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:40.590034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:40.658617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:40.660331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2670b-1f1c-47b8-8408-ab430551eddc service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:40.683902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 666/1311] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:43.211401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:43.211910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:43.211910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:43.213247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 646/1312] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:43.218501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b223-56c1-4935-a9ff-741ca204402c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:43.218992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b223-56c1-4935-a9ff-741ca204402c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:43.219098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5b223-56c1-4935-a9ff-741ca204402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:43.299085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d0c5b223-56c1-4935-a9ff-741ca204402c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:43.300284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d0c5b223-56c1-4935-a9ff-741ca204402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:43.396724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 667/1313] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:43.434942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:43.436651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:43.542965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724cb18-14b1-4263-aea6-cf213f032951 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:43.621623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:43.624362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:43.625008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:43.625293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:43.625566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:43.627298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:43.743390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:43.744833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21280ad-6d64-40ca-b158-8ab8cc219d61 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:43.764550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 647/1314] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:46.522568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:46.523149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:46.523149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:46.524522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 668/1315] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:46.529777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d71b2-51f6-4cb2-92a8-28b267fadd10 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:46.530211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d71b2-51f6-4cb2-92a8-28b267fadd10 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:46.530459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d71b2-51f6-4cb2-92a8-28b267fadd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:46.598139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d32d71b2-51f6-4cb2-92a8-28b267fadd10 None None] MFA Rules not processed for user `5176609c476e4288aae0b9edac826d58`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:46.599319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d32d71b2-51f6-4cb2-92a8-28b267fadd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:46.651271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:46.697093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 648/1316] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:46.737041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:46.738664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:46.748437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:46.859160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081637d-2b4f-4adc-b57b-7e9b2394bc7d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:46.936202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:46.936854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:46.937134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:46.937414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:46.939786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:47.004999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:47.052336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:47.053794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dee43-5a4c-4ad9-993e-8dc0825cff18 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:47.072342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 669/1317] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:50.035406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:50.035953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:50.035953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:50.037338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 649/1318] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:50.042498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c321f4-aea0-4a39-bff8-d6a673ccd8a0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:50.043022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c321f4-aea0-4a39-bff8-d6a673ccd8a0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:50.043260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c321f4-aea0-4a39-bff8-d6a673ccd8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:50.108191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-42c321f4-aea0-4a39-bff8-d6a673ccd8a0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:50.109412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-42c321f4-aea0-4a39-bff8-d6a673ccd8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:50.207748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 670/1319] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:50.245574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:50.247169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:50.353096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a8686-d4bb-4daa-abc7-853feb5077cc None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:50.425705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:50.426355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:50.426663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:50.426943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:50.428601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:50.436265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:50.538629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:50.540277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c09772-105f-4b4b-a13a-59a2d4752791 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:50.559211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 650/1320] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:53.810036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa82dc-5591-4ca0-8458-35496cda7ccb None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:53.810667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa82dc-5591-4ca0-8458-35496cda7ccb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:53.810667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa82dc-5591-4ca0-8458-35496cda7ccb None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:53.811983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 671/1321] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:53.817289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511bcdf-c1b0-4536-a91f-df07f4e2386c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:53.817733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511bcdf-c1b0-4536-a91f-df07f4e2386c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:53.817833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511bcdf-c1b0-4536-a91f-df07f4e2386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:53.882698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d511bcdf-c1b0-4536-a91f-df07f4e2386c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:53.883925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d511bcdf-c1b0-4536-a91f-df07f4e2386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:53.941021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:53.983016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 651/1322] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:54.033733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:54.034204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:54.034486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:54.035960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 672/1323] 10.222.0.74 () {54 vars in 964 bytes} [Wed May 25 14:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:54.042227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da053-39d5-4921-9c3d-08803f72ac52 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:54.042583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da053-39d5-4921-9c3d-08803f72ac52 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:54.043009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8da053-39d5-4921-9c3d-08803f72ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:54.118961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ae8da053-39d5-4921-9c3d-08803f72ac52 None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:54.120087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ae8da053-39d5-4921-9c3d-08803f72ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:54.194408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 652/1324] 10.222.0.74 () {58 vars in 1061 bytes} [Wed May 25 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:54.203688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:54.205811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:54.225687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:54.310031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c58d1a-6e54-4219-9638-123d150edc88 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:54.384670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:54.385383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:54.385672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:54.385950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:54.387632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:54.462655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:54.502070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:54.503600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6721cc-a46a-4c2f-8827-02d6e60ee639 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:54.522146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 673/1325] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:57.051773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:57.052312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:57.052312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:57.053705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 653/1326] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:57.059233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd63a6-b062-4be1-95b4-4078ddb442e5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:57.059643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd63a6-b062-4be1-95b4-4078ddb442e5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:57.059802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd63a6-b062-4be1-95b4-4078ddb442e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:57.125454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4fcd63a6-b062-4be1-95b4-4078ddb442e5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:57.127275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4fcd63a6-b062-4be1-95b4-4078ddb442e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:57.222739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 674/1327] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:57.259375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:57.260910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:57.325731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:57.369036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad9ce-16de-4fec-8742-c898acb4158f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:57.447048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:57.447696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:57.447977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:57.448249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:57.449870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:57.558193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:57.560662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:57.562098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3642c-ef93-4149-a666-7ecbc4c09ebc service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:57.582396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 654/1328] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:59.859900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:59.861150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:59.861150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:59.861858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 675/1329] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:59.867662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156343cb-43ed-4b6e-b6e1-af5d0f0d8fcd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:59.868402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156343cb-43ed-4b6e-b6e1-af5d0f0d8fcd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:59.868578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156343cb-43ed-4b6e-b6e1-af5d0f0d8fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:59.940521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-156343cb-43ed-4b6e-b6e1-af5d0f0d8fcd None None] MFA Rules not processed for user `5176609c476e4288aae0b9edac826d58`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:59.941806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-156343cb-43ed-4b6e-b6e1-af5d0f0d8fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:00.057325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 655/1330] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:00.095821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:00.097781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:00.131143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:00.219505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24d317-f1a3-4753-87a3-5c8838530c28 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:00.297066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:00.298214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:00.298493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:00.298789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:00.300433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:00.390306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:00.433645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:00.435150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ab68-4663-461d-b5af-5772535a0571 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:00.458629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 676/1331] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:03.117656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:03.120768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:03.120768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:03.120768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 656/1332] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:03.126361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506b10f-986b-4d42-acfa-35166e045eb9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:03.126921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506b10f-986b-4d42-acfa-35166e045eb9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:03.127027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506b10f-986b-4d42-acfa-35166e045eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:03.198875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1506b10f-986b-4d42-acfa-35166e045eb9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:03.200109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1506b10f-986b-4d42-acfa-35166e045eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:03.316978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 677/1333] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:03.353691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:03.355241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:03.426316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:03.460581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21a378-b850-47c9-934e-349bdf40d3bb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:03.540037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:03.540684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:03.540965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:03.541239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:03.542932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:03.666460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:03.667982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697fb2-92c3-4f9d-a63b-5552a4b9ae3a service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:03.680424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:03.690248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 657/1334] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:05.981136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:05.981716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:05.981716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:05.983147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 678/1335] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:05.989168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224422d3-1985-491b-87ee-1907037f6c29 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:05.989633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224422d3-1985-491b-87ee-1907037f6c29 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:05.989743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224422d3-1985-491b-87ee-1907037f6c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:06.054369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-224422d3-1985-491b-87ee-1907037f6c29 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:06.055629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-224422d3-1985-491b-87ee-1907037f6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:06.153796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 658/1336] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:06.193647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:06.195224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:06.236290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:06.299559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aa0f6-ee72-42ce-8b35-f1cbf734995f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:06.383504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:06.384155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:06.384444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:06.384725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:06.386378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:06.471779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:06.499950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:06.502545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e2916-c7c4-4c25-b1b7-b9e93a951677 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:06.522804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 679/1337] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:09.686950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:09.687475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:09.687475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:09.688883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 659/1338] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:09.694493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09953b-3b5c-443c-a3e0-2d347e8a3a52 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:09.694956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09953b-3b5c-443c-a3e0-2d347e8a3a52 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:09.695096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09953b-3b5c-443c-a3e0-2d347e8a3a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:09.759234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0d09953b-3b5c-443c-a3e0-2d347e8a3a52 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:09.760450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0d09953b-3b5c-443c-a3e0-2d347e8a3a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:09.857445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 680/1339] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:09.893468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:09.896608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:09.977941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:10.007578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce379-4930-4c4b-a2f3-dc4265902bb8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:10.021374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:10.022022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:10.022303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:10.022577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:10.024221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:10.134467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:10.135983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db2cb1-4158-4233-962a-e1045ebc4617 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:10.159301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 660/1340] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:14.864306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:14.864879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:14.864879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:14.866244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 681/1341] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:14.872612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195539ba-ed0a-4171-9059-71906d04e00e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:14.873146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195539ba-ed0a-4171-9059-71906d04e00e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:14.873253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195539ba-ed0a-4171-9059-71906d04e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:14.933673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-195539ba-ed0a-4171-9059-71906d04e00e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:14.937279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-195539ba-ed0a-4171-9059-71906d04e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:15.004914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:15.038479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 661/1342] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:15.079764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:15.082100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:15.113043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c7aa7-2368-45f6-b896-c1729adad092 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:15.118455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:15.118455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:15.118455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:15.118455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:15.128294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:15.129900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b0189-75dd-43f1-95f0-d04b41be971e service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:15.146693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 682/1343] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:18.370333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae59a-5f65-4bba-a588-617fc52d425a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:18.370890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae59a-5f65-4bba-a588-617fc52d425a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:18.371012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ae59a-5f65-4bba-a588-617fc52d425a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:18.372323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 662/1344] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:18.377538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727fda7-9dad-4794-8194-59902af9d7a9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:18.377973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727fda7-9dad-4794-8194-59902af9d7a9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:18.378227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727fda7-9dad-4794-8194-59902af9d7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:18.409147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d727fda7-9dad-4794-8194-59902af9d7a9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:18.410355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d727fda7-9dad-4794-8194-59902af9d7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:18.418715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 683/1345] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:18.456614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ae59a-5f65-4bba-a588-617fc52d425a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:18.468771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ae59a-5f65-4bba-a588-617fc52d425a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:18.472554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:18.473200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:18.473487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:18.473766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:18.483760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:18.485243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee4d48-6017-4e8f-82b2-dc7d1f3060cb service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:18.495819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 663/1346] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:22.906857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc717-0898-4450-b17a-98813e188383 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:22.907382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc717-0898-4450-b17a-98813e188383 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:22.907502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc717-0898-4450-b17a-98813e188383 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:22.908803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 684/1347] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:22.913994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba60d38-cd90-42ff-b402-a343b2dbe491 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:22.914388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba60d38-cd90-42ff-b402-a343b2dbe491 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:22.914699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba60d38-cd90-42ff-b402-a343b2dbe491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:22.932800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1ba60d38-cd90-42ff-b402-a343b2dbe491 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:22.934009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1ba60d38-cd90-42ff-b402-a343b2dbe491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:22.942608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 664/1348] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:22.980000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dc717-0898-4450-b17a-98813e188383 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:22.991007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dc717-0898-4450-b17a-98813e188383 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:22.994586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:22.995242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:22.995526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:22.995802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:23.002085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:23.005608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:23.015479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec381770-5883-4611-a815-747d3fa133ec service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:23.031206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 685/1349] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:28.384080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eea40-85a6-4636-b052-821c6be9aa3c None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:28.384598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eea40-85a6-4636-b052-821c6be9aa3c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:28.384705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eea40-85a6-4636-b052-821c6be9aa3c None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:28.386380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 665/1350] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:28.391869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74cc1-1540-4dca-a3cd-dad1ce70e450 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:28.392712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74cc1-1540-4dca-a3cd-dad1ce70e450 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:28.393231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb74cc1-1540-4dca-a3cd-dad1ce70e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:28.521525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-feb74cc1-1540-4dca-a3cd-dad1ce70e450 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:28.523320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-feb74cc1-1540-4dca-a3cd-dad1ce70e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:28.604665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 686/1351] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:32.927303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=19 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:32.988888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4c0c-f4e8-4ef5-8b8c-162be673a662 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:32.989423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4c0c-f4e8-4ef5-8b8c-162be673a662 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:32.989718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4c0c-f4e8-4ef5-8b8c-162be673a662 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:32.991151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 666/1352] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:32.997104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b0e71-77e9-4fa7-b352-7ecb154c5627 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:32.997656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b0e71-77e9-4fa7-b352-7ecb154c5627 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:32.997656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b0e71-77e9-4fa7-b352-7ecb154c5627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:33.018335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2f7b0e71-77e9-4fa7-b352-7ecb154c5627 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:33.019921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2f7b0e71-77e9-4fa7-b352-7ecb154c5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:33.029163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 687/1353] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:34.583506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd6af7-fcd6-4b1c-8dbc-ceab8f6bf99a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:34.584015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd6af7-fcd6-4b1c-8dbc-ceab8f6bf99a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:34.584015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd6af7-fcd6-4b1c-8dbc-ceab8f6bf99a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:34.585836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 667/1354] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:34.590992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9068-d503-4e6b-a4e7-862b47fa767a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:34.591448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9068-d503-4e6b-a4e7-862b47fa767a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:34.591673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9068-d503-4e6b-a4e7-862b47fa767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:34.610832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e3be9068-d503-4e6b-a4e7-862b47fa767a None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:34.612123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e3be9068-d503-4e6b-a4e7-862b47fa767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:34.620615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 688/1355] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:46:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:38.658378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40fbf8-6c03-4bf3-be80-24d1c11106a2 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:38.658946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40fbf8-6c03-4bf3-be80-24d1c11106a2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:38.658946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40fbf8-6c03-4bf3-be80-24d1c11106a2 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:38.660318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 668/1356] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:38.665571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03574-1069-4249-a85e-57f7fa20ff96 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:38.665996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03574-1069-4249-a85e-57f7fa20ff96 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:38.666129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03574-1069-4249-a85e-57f7fa20ff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:38.686222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c0b03574-1069-4249-a85e-57f7fa20ff96 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:38.688383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c0b03574-1069-4249-a85e-57f7fa20ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:38.696600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 689/1357] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:38.753805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bfba-66bd-4222-bdea-f58f16276278 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:38.754083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bfba-66bd-4222-bdea-f58f16276278 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:38.755341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bfba-66bd-4222-bdea-f58f16276278 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:38.759534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 669/1358] 10.222.0.74 () {54 vars in 961 bytes} [Wed May 25 14:46:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:38.776226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1bde-8704-4274-a90a-f4204b9ae72a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:38.776918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1bde-8704-4274-a90a-f4204b9ae72a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:38.777430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1bde-8704-4274-a90a-f4204b9ae72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:38.853670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ef4c1bde-8704-4274-a90a-f4204b9ae72a None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:38.854916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ef4c1bde-8704-4274-a90a-f4204b9ae72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:38.911557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:38.956027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 690/1359] 10.222.0.74 () {58 vars in 1058 bytes} [Wed May 25 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:38.964743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1bfba-66bd-4222-bdea-f58f16276278 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:38.980362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1bfba-66bd-4222-bdea-f58f16276278 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:38.987399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:38.988134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:38.988612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:38.989062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:38.999339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:39.001692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e730d-c8d5-4876-b9e0-8fbd925d7583 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:39.014960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 670/1360] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:41.492579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357b264-803b-45ab-8c60-dcef55d22d57 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:41.493103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357b264-803b-45ab-8c60-dcef55d22d57 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:41.493202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357b264-803b-45ab-8c60-dcef55d22d57 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:41.494694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 691/1361] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:41.500526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2aaba-ed7e-4232-a1af-95d746c781f6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:41.501680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2aaba-ed7e-4232-a1af-95d746c781f6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:41.502573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2aaba-ed7e-4232-a1af-95d746c781f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:41.522559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2bb2aaba-ed7e-4232-a1af-95d746c781f6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:41.523910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2bb2aaba-ed7e-4232-a1af-95d746c781f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:41.532014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 671/1362] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:41.576539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357b264-803b-45ab-8c60-dcef55d22d57 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:41.591359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357b264-803b-45ab-8c60-dcef55d22d57 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:41.597236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:41.597986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:41.598381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:41.598802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:41.607714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:41.609973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b5d1e-bdc0-4d94-8fe3-e2a8e599ec30 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:41.620612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 692/1363] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:44.280618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9502a-f44b-4df7-a841-592bd766a657 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.281213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9502a-f44b-4df7-a841-592bd766a657 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.281307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9502a-f44b-4df7-a841-592bd766a657 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.282756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 672/1364] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:44.293649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9020a9-303b-43f3-ab19-0076e611cee8 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.294065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9020a9-303b-43f3-ab19-0076e611cee8 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.294215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9020a9-303b-43f3-ab19-0076e611cee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.317326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0c9020a9-303b-43f3-ab19-0076e611cee8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:44.324831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0c9020a9-303b-43f3-ab19-0076e611cee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:44.349746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 693/1365] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:44.396988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57441bd-ccc0-4891-bfe6-2d4a29e7db25 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.397517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57441bd-ccc0-4891-bfe6-2d4a29e7db25 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.397907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57441bd-ccc0-4891-bfe6-2d4a29e7db25 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.399469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 673/1366] 10.222.0.74 () {54 vars in 961 bytes} [Wed May 25 14:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:44.404786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1094-6f9b-4aee-88ee-c924d90a8c72 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.406292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1094-6f9b-4aee-88ee-c924d90a8c72 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.406292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1094-6f9b-4aee-88ee-c924d90a8c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.424629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-481a1094-6f9b-4aee-88ee-c924d90a8c72 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:44.425829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-481a1094-6f9b-4aee-88ee-c924d90a8c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:44.435383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 694/1367] 10.222.0.74 () {58 vars in 1058 bytes} [Wed May 25 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:44.444359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57441bd-ccc0-4891-bfe6-2d4a29e7db25 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:44.460250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57441bd-ccc0-4891-bfe6-2d4a29e7db25 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:44.465529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:44.466161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.466442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.466774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.475334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:44.476933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548eb20-b183-4157-997c-2993067f2068 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:44.486760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 674/1368] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:46.909515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5018-61e4-44f2-b206-9bc362f90921 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:46.910068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5018-61e4-44f2-b206-9bc362f90921 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:46.910068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd5018-61e4-44f2-b206-9bc362f90921 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:46.911757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 695/1369] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:46.917584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfae2a-83c2-43e0-8853-a6977283cd27 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:46.918108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfae2a-83c2-43e0-8853-a6977283cd27 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:46.918208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfae2a-83c2-43e0-8853-a6977283cd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:46.938950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dfdfae2a-83c2-43e0-8853-a6977283cd27 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:46.940368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dfdfae2a-83c2-43e0-8853-a6977283cd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:46.949535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 675/1370] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:46.994571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd5018-61e4-44f2-b206-9bc362f90921 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:47.009170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd5018-61e4-44f2-b206-9bc362f90921 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:47.014884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:47.015580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.015870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.016154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.025334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:47.026991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea7672-7ddc-4381-9af9-c0cc1d24a84d service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:47.037947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 696/1371] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:47.055423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea3f83-ba4c-4854-bcd9-0cc9eb1eb5ab None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.055711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea3f83-ba4c-4854-bcd9-0cc9eb1eb5ab None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.055994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea3f83-ba4c-4854-bcd9-0cc9eb1eb5ab None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.057628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 676/1372] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:47.063502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15795b6f-8227-4d41-821d-7b95cece2596 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.073059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15795b6f-8227-4d41-821d-7b95cece2596 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.073059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15795b6f-8227-4d41-821d-7b95cece2596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.084987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-15795b6f-8227-4d41-821d-7b95cece2596 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:47.086278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-15795b6f-8227-4d41-821d-7b95cece2596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.095256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 697/1373] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:47.428450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347925-6534-4291-8878-068a3102256a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.429020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347925-6534-4291-8878-068a3102256a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.429441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e347925-6534-4291-8878-068a3102256a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.431107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 677/1374] 10.222.0.74 () {54 vars in 963 bytes} [Wed May 25 14:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:47.438323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1660-4abc-4d62-a645-ea529f7b2d1e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.438609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1660-4abc-4d62-a645-ea529f7b2d1e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.438976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1660-4abc-4d62-a645-ea529f7b2d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.492138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-650e1660-4abc-4d62-a645-ea529f7b2d1e None None] MFA Rules not processed for user `ffea1035a1d2434e948256087a112b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:47.493362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-650e1660-4abc-4d62-a645-ea529f7b2d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.559026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 698/1375] 10.222.0.74 () {58 vars in 1060 bytes} [Wed May 25 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:47.566439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e347925-6534-4291-8878-068a3102256a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:47.579369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e347925-6534-4291-8878-068a3102256a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:47.583005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:47.583736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.584116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.584465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.595300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:47.596837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7c60-21df-4220-bc95-869ec584c679 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:47.607151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 678/1376] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:49.449840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4fef-f551-42cb-b437-c48baabf4e26 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:49.450393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4fef-f551-42cb-b437-c48baabf4e26 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:49.450393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4fef-f551-42cb-b437-c48baabf4e26 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:49.452127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 699/1377] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:49.458159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e82b-13d5-4a1d-b083-bbda1b8efeab None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:49.458745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e82b-13d5-4a1d-b083-bbda1b8efeab None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:49.458865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e82b-13d5-4a1d-b083-bbda1b8efeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:49.478286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d3a4e82b-13d5-4a1d-b083-bbda1b8efeab None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:49.479568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d3a4e82b-13d5-4a1d-b083-bbda1b8efeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:49.488309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 679/1378] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:49.534020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4fef-f551-42cb-b437-c48baabf4e26 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:49.549694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4fef-f551-42cb-b437-c48baabf4e26 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:49.555469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:49.556365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:49.556659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:49.557012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:49.566600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:49.568285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faaaa-3b71-444f-a48a-2480350e764f service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:49.579649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 700/1379] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:51.964359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539815-1e1d-4857-b362-52f03eeb23c6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:51.965236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539815-1e1d-4857-b362-52f03eeb23c6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:51.965666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539815-1e1d-4857-b362-52f03eeb23c6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:51.968871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 680/1380] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:51.972945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b6b6-87ab-4de1-91e4-f4ad4877789a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:51.974083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b6b6-87ab-4de1-91e4-f4ad4877789a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:51.974442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b6b6-87ab-4de1-91e4-f4ad4877789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:51.994132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3069b6b6-87ab-4de1-91e4-f4ad4877789a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:51.995806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3069b6b6-87ab-4de1-91e4-f4ad4877789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:52.005777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 701/1381] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:52.049555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce539815-1e1d-4857-b362-52f03eeb23c6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:52.064318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce539815-1e1d-4857-b362-52f03eeb23c6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:52.069739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:52.070375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:52.070681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:52.070963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:52.080147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:52.081724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa11aad-78d7-44b7-af00-0b67176a57b0 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:52.091866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 681/1382] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:54.482317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8d64-9a6a-44c2-8ea8-c2b80003172b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:54.482960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8d64-9a6a-44c2-8ea8-c2b80003172b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:54.482960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8d64-9a6a-44c2-8ea8-c2b80003172b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:54.484883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 702/1383] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:54.490858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b16fe-97ff-4b17-bb94-521f91c8e7c2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:54.491388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b16fe-97ff-4b17-bb94-521f91c8e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:54.491388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b16fe-97ff-4b17-bb94-521f91c8e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:54.511314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-057b16fe-97ff-4b17-bb94-521f91c8e7c2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:54.512801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-057b16fe-97ff-4b17-bb94-521f91c8e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:54.521102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 682/1384] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:54.562983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8d64-9a6a-44c2-8ea8-c2b80003172b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:54.578548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8d64-9a6a-44c2-8ea8-c2b80003172b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:54.584112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:54.584860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:54.585247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:54.585624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:54.595567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:54.597274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902784e-d466-468c-bd57-bb13a1ce97cb service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:54.608057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 703/1385] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:56.844743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698d4e-4a92-4a5c-b4f8-1fadc5cb6121 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:56.845359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698d4e-4a92-4a5c-b4f8-1fadc5cb6121 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:56.845359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698d4e-4a92-4a5c-b4f8-1fadc5cb6121 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:56.846686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 683/1386] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:56.852001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc3ac-ac86-437e-b841-2d08f37c8d58 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:56.852447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc3ac-ac86-437e-b841-2d08f37c8d58 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:56.852608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc3ac-ac86-437e-b841-2d08f37c8d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:56.871963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b63fc3ac-ac86-437e-b841-2d08f37c8d58 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:56.873184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b63fc3ac-ac86-437e-b841-2d08f37c8d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:56.881587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 704/1387] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:56.926884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698d4e-4a92-4a5c-b4f8-1fadc5cb6121 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:56.942062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698d4e-4a92-4a5c-b4f8-1fadc5cb6121 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:56.948085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:56.948723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:56.949066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:56.949287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:56.958438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:56.960093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519491ad-c8a1-48a9-991c-c9731e5d4c43 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:56.970229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 684/1388] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:59.392988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf046-c473-40f9-93d5-babcf48cf780 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:59.393540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf046-c473-40f9-93d5-babcf48cf780 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:59.393787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf046-c473-40f9-93d5-babcf48cf780 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:59.395596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 705/1389] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:59.400996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0cf1-024b-46bb-930e-3af4c3cc730d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:59.401429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0cf1-024b-46bb-930e-3af4c3cc730d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:59.401548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0cf1-024b-46bb-930e-3af4c3cc730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:59.420168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fcac0cf1-024b-46bb-930e-3af4c3cc730d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:59.421917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fcac0cf1-024b-46bb-930e-3af4c3cc730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:59.431178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 685/1390] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:59.483081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbf046-c473-40f9-93d5-babcf48cf780 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:59.502929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbf046-c473-40f9-93d5-babcf48cf780 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:59.510054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:59.511150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:59.511602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:59.511961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:59.523254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:59.525321 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98870199-7f6d-4d3d-9abd-8e1c02a7d318 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:59.538532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 706/1391] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:01.951255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2621e-d5f4-499f-950a-7b1c7efc3c6f None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:01.951901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2621e-d5f4-499f-950a-7b1c7efc3c6f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:01.951901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2621e-d5f4-499f-950a-7b1c7efc3c6f None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:01.953762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 686/1392] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:01.959070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22ff3-9e1d-4828-bfe5-8f9d6636323e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:01.959501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22ff3-9e1d-4828-bfe5-8f9d6636323e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:01.959635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22ff3-9e1d-4828-bfe5-8f9d6636323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:01.979920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2fb22ff3-9e1d-4828-bfe5-8f9d6636323e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:01.981131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2fb22ff3-9e1d-4828-bfe5-8f9d6636323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:01.989573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 707/1393] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:02.034350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2621e-d5f4-499f-950a-7b1c7efc3c6f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:02.049344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2621e-d5f4-499f-950a-7b1c7efc3c6f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:02.054802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:02.055442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:02.055729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:02.056005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:02.065039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:02.066622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06f6e0-7a64-40bd-ac89-4b33195d3e8b service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:02.076425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 687/1394] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:04.041172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb7230-c53c-4d7e-a8b4-76e6bd82581d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.041739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb7230-c53c-4d7e-a8b4-76e6bd82581d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.041739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb7230-c53c-4d7e-a8b4-76e6bd82581d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.043122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 708/1395] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:47:04.048797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb038-1757-4908-9d8f-eb2baa6e616e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.049198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb038-1757-4908-9d8f-eb2baa6e616e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.049352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb038-1757-4908-9d8f-eb2baa6e616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.055385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=52 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:04.068322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a74cb038-1757-4908-9d8f-eb2baa6e616e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:04.069527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a74cb038-1757-4908-9d8f-eb2baa6e616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.078307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 688/1396] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:04.134581 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65be90-a111-4609-a13f-1e574df246c2 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.134927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65be90-a111-4609-a13f-1e574df246c2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.135204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65be90-a111-4609-a13f-1e574df246c2 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.136591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 709/1397] 10.222.0.74 () {54 vars in 962 bytes} [Wed May 25 14:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:04.142078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead60-86a1-4adf-a21a-6f7301e9109d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.142373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead60-86a1-4adf-a21a-6f7301e9109d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.142678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdead60-86a1-4adf-a21a-6f7301e9109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.198862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dbdead60-86a1-4adf-a21a-6f7301e9109d None None] MFA Rules not processed for user `6cdeab1ae06e4df9a270a716ae0a1db3`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:04.200085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dbdead60-86a1-4adf-a21a-6f7301e9109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.264991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 689/1398] 10.222.0.74 () {58 vars in 1059 bytes} [Wed May 25 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:04.273018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65be90-a111-4609-a13f-1e574df246c2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:04.278657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:04.285156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65be90-a111-4609-a13f-1e574df246c2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:04.289372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:04.290035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.290319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.290597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.301821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:04.303698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab298a-441b-425f-8f9b-0b5762bcaf56 service glance] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:04.316072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 710/1399] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:04.479108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031adf9-385a-4316-8837-a0ede9d27d20 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.479457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031adf9-385a-4316-8837-a0ede9d27d20 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.479683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031adf9-385a-4316-8837-a0ede9d27d20 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.481116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 690/1400] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:04.486906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6623-ca4a-463d-a118-77be620f0394 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.487200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6623-ca4a-463d-a118-77be620f0394 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.487478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6623-ca4a-463d-a118-77be620f0394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.509306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0f1b6623-ca4a-463d-a118-77be620f0394 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:04.511319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0f1b6623-ca4a-463d-a118-77be620f0394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.520959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 711/1401] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:04.570623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031adf9-385a-4316-8837-a0ede9d27d20 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:04.587405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031adf9-385a-4316-8837-a0ede9d27d20 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:04.593136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:04.593780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.594064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.594338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.603187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:04.604794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df0f19-3fee-48b2-a80f-6cbfe0b587d4 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:04.615310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 691/1402] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:47:05.026006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828576eb-bd9f-459a-bab8-e7e68f69bab2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:05.026310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828576eb-bd9f-459a-bab8-e7e68f69bab2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:05.026595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828576eb-bd9f-459a-bab8-e7e68f69bab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:05.076966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-828576eb-bd9f-459a-bab8-e7e68f69bab2 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:05.078506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-828576eb-bd9f-459a-bab8-e7e68f69bab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:05.152280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 712/1403] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:05.199512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8bf2c-44db-4f77-ba77-24229273f61e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:05.200045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8bf2c-44db-4f77-ba77-24229273f61e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:05.200145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8bf2c-44db-4f77-ba77-24229273f61e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:05.201779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 692/1404] 10.222.0.74 () {54 vars in 942 bytes} [Wed May 25 14:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:05.210934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6680a-8a55-4629-9041-3eb462754968 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:05.211229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6680a-8a55-4629-9041-3eb462754968 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:05.211512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6680a-8a55-4629-9041-3eb462754968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:05.261943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-deb6680a-8a55-4629-9041-3eb462754968 None None] MFA Rules not processed for user `66a7eff73b85406a8874547263816a68`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:05.263264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-deb6680a-8a55-4629-9041-3eb462754968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:05.345074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 713/1405] 10.222.0.74 () {58 vars in 1039 bytes} [Wed May 25 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:05.354392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8bf2c-44db-4f77-ba77-24229273f61e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:05.368588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8bf2c-44db-4f77-ba77-24229273f61e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:05.373893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:05.375198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:05.375549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:05.375893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:05.384581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:05.386152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb429cb-0c2b-4fdd-a34f-4aea94f9427c service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:05.394206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 693/1406] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:47:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:47:07.036230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9e6ae-2383-4730-988f-b658947543e1 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.036814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9e6ae-2383-4730-988f-b658947543e1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.036814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9e6ae-2383-4730-988f-b658947543e1 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.038194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 714/1407] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:07.043775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c57cb-d1df-4a60-8b79-3cf935a654c6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.044227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c57cb-d1df-4a60-8b79-3cf935a654c6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.044318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c57cb-d1df-4a60-8b79-3cf935a654c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.069422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b07c57cb-d1df-4a60-8b79-3cf935a654c6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:07.070673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b07c57cb-d1df-4a60-8b79-3cf935a654c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:07.086234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 694/1408] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:07.133260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9e6ae-2383-4730-988f-b658947543e1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:07.149385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9e6ae-2383-4730-988f-b658947543e1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:07.155217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:07.156009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.156355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.156700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.167098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:07.168857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b103-04eb-4aa7-9c84-e307b7de0663 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:07.181655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 715/1409] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:17.057956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=55 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:17.176609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:29.911635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d206-9253-4703-95af-2249b0154fd8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.912279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d206-9253-4703-95af-2249b0154fd8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.912279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d206-9253-4703-95af-2249b0154fd8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.913563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 695/1410] 10.222.0.74 () {54 vars in 918 bytes} [Wed May 25 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:29.918476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebdfc7-e44f-4711-8142-ca3ced806061 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.918925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebdfc7-e44f-4711-8142-ca3ced806061 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.919055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebdfc7-e44f-4711-8142-ca3ced806061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.937236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-24ebdfc7-e44f-4711-8142-ca3ced806061 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:29.938418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-24ebdfc7-e44f-4711-8142-ca3ced806061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:29.947201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 716/1411] 10.222.0.74 () {58 vars in 1015 bytes} [Wed May 25 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:29.961018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5966-3cf7-4e40-b9b1-0227c276cbd6 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.961301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5966-3cf7-4e40-b9b1-0227c276cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.961575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd5966-3cf7-4e40-b9b1-0227c276cbd6 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.962960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 696/1412] 10.222.0.74 () {54 vars in 918 bytes} [Wed May 25 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:29.968060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c2558-24dd-4f71-9d36-a978b7a7ffc4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.968343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c2558-24dd-4f71-9d36-a978b7a7ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.968621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c2558-24dd-4f71-9d36-a978b7a7ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.986404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4c0c2558-24dd-4f71-9d36-a978b7a7ffc4 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:29.987603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4c0c2558-24dd-4f71-9d36-a978b7a7ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:29.996798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 717/1413] 10.222.0.74 () {58 vars in 1015 bytes} [Wed May 25 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:32.644982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ca0e-713a-4239-8009-f7a49078d19a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:32.645520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ca0e-713a-4239-8009-f7a49078d19a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:32.645520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ca0e-713a-4239-8009-f7a49078d19a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:32.646833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:32.652024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23597-324c-4c05-8221-353e120a605e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:32.652460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23597-324c-4c05-8221-353e120a605e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:32.652616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23597-324c-4c05-8221-353e120a605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:32.670527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-78b23597-324c-4c05-8221-353e120a605e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:32.671765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-78b23597-324c-4c05-8221-353e120a605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:32.679572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:47:32.722609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ca0e-713a-4239-8009-f7a49078d19a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:32.737083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ca0e-713a-4239-8009-f7a49078d19a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:32.742348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:32.743008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:32.743296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:32.743571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:32.752685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:32.754258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d47fe-e955-4dcd-a32e-9e5d70fc23ca service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:32.763613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 698/1416] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:33.733175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa3dc-a6b3-47d8-a4c7-3fee2a760775 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:33.733698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa3dc-a6b3-47d8-a4c7-3fee2a760775 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:33.733818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afa3dc-a6b3-47d8-a4c7-3fee2a760775 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:33.735142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:33.740213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fab9f-4689-4f64-9b3b-c39d4ecb6138 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:33.740664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fab9f-4689-4f64-9b3b-c39d4ecb6138 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:33.740779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fab9f-4689-4f64-9b3b-c39d4ecb6138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:33.759090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0d0fab9f-4689-4f64-9b3b-c39d4ecb6138 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:33.760272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0d0fab9f-4689-4f64-9b3b-c39d4ecb6138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:33.768388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:33.929189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afa3dc-a6b3-47d8-a4c7-3fee2a760775 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:33.944154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afa3dc-a6b3-47d8-a4c7-3fee2a760775 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:33.949550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:33.950429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:33.950739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:33.951019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:33.959047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:33.960633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2177608-87c4-40c1-81b3-5c14fe063658 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:33.969943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 720/1419] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:34.892758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcefa-2661-4636-82f9-0d8ebbda9c9a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:34.893318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcefa-2661-4636-82f9-0d8ebbda9c9a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:34.893318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfcefa-2661-4636-82f9-0d8ebbda9c9a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:34.894724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:34.899521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f049-9104-4049-aa1b-2b50f29abcc5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:34.899967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f049-9104-4049-aa1b-2b50f29abcc5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:34.900073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3f049-9104-4049-aa1b-2b50f29abcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:34.918448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4bd3f049-9104-4049-aa1b-2b50f29abcc5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:34.919666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4bd3f049-9104-4049-aa1b-2b50f29abcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:34.927250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:35.079118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfcefa-2661-4636-82f9-0d8ebbda9c9a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:35.093401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfcefa-2661-4636-82f9-0d8ebbda9c9a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:35.098892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:35.099524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:35.099811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:35.100090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:35.108106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:35.109688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dc345-368e-4172-9738-6e9799094d45 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:35.119031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 701/1422] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:36.022317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3a8e0-08a1-4c51-886a-676e57527592 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:36.022923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3a8e0-08a1-4c51-886a-676e57527592 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:36.022923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3a8e0-08a1-4c51-886a-676e57527592 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:36.024298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:36.029261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd80560-b86b-484d-9bb9-c308e36ababa None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:36.030167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd80560-b86b-484d-9bb9-c308e36ababa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:36.030344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd80560-b86b-484d-9bb9-c308e36ababa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:36.048437 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ddd80560-b86b-484d-9bb9-c308e36ababa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:36.049631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ddd80560-b86b-484d-9bb9-c308e36ababa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:36.057317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:36.208153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3a8e0-08a1-4c51-886a-676e57527592 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:36.223112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3a8e0-08a1-4c51-886a-676e57527592 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:36.228471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:36.229100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:36.229386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:36.229664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:36.237617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:36.239267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf70-c298-4599-8531-c0f58c858293 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:36.248586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 723/1425] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:37.157003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090faf-f6d2-426e-b5a3-eb3f2970a01b None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:37.157641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090faf-f6d2-426e-b5a3-eb3f2970a01b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:37.157641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090faf-f6d2-426e-b5a3-eb3f2970a01b None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:37.159016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:37.163860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e231d8-6872-43b6-8b92-6faa5f58bc35 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:37.164259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e231d8-6872-43b6-8b92-6faa5f58bc35 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:37.164415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e231d8-6872-43b6-8b92-6faa5f58bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:37.183172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f5e231d8-6872-43b6-8b92-6faa5f58bc35 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:37.184412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f5e231d8-6872-43b6-8b92-6faa5f58bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:37.192838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:37.341386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05090faf-f6d2-426e-b5a3-eb3f2970a01b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:37.355567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05090faf-f6d2-426e-b5a3-eb3f2970a01b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:37.361214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:37.361980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:37.362260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:37.362534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:37.370807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:37.372399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fe3fc-8daf-4dcc-902f-00df17713704 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:37.382447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 704/1428] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:38.310901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6370e-2d97-4436-af7e-d56bca85654d None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:38.311362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6370e-2d97-4436-af7e-d56bca85654d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:38.311546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6370e-2d97-4436-af7e-d56bca85654d None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:38.313344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:38.318652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dec7-6190-478e-b04d-cce8a14f4edc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:38.319163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dec7-6190-478e-b04d-cce8a14f4edc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:38.319454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dec7-6190-478e-b04d-cce8a14f4edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:38.338117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9df2dec7-6190-478e-b04d-cce8a14f4edc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:38.339345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9df2dec7-6190-478e-b04d-cce8a14f4edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:38.347860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:38.496801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6370e-2d97-4436-af7e-d56bca85654d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:38.511535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6370e-2d97-4436-af7e-d56bca85654d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:38.517626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:38.518280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:38.518574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:38.518887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:38.526975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:38.528560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e6a5-3be2-4479-b3a3-074918f99f5c service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:38.538384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 726/1431] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:39.437766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4580-dbfc-4e0a-a53b-eea45405c77a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:39.438300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4580-dbfc-4e0a-a53b-eea45405c77a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:39.438300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4580-dbfc-4e0a-a53b-eea45405c77a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:39.439717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:39.444738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06011-92df-426d-b34c-ef392c1b8037 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:39.445169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06011-92df-426d-b34c-ef392c1b8037 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:39.445355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06011-92df-426d-b34c-ef392c1b8037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:39.464174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7bd06011-92df-426d-b34c-ef392c1b8037 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:39.465372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7bd06011-92df-426d-b34c-ef392c1b8037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:39.473633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:39.620728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c4580-dbfc-4e0a-a53b-eea45405c77a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:39.634692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c4580-dbfc-4e0a-a53b-eea45405c77a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:39.639930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:39.640554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:39.640837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:39.641111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:39.648972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:39.650529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9596206-8dc5-4056-aab2-fcb90dc0d904 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:39.660135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 707/1434] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:40.585272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5272b-9346-4faa-80d8-cca58fa8b6fd None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.585820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5272b-9346-4faa-80d8-cca58fa8b6fd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.585820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5272b-9346-4faa-80d8-cca58fa8b6fd None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.587227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:40.592253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cb349-f08b-41b3-95c4-2b830d2d4073 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.592669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cb349-f08b-41b3-95c4-2b830d2d4073 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.592821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cb349-f08b-41b3-95c4-2b830d2d4073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.610869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2f3cb349-f08b-41b3-95c4-2b830d2d4073 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:40.612058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2f3cb349-f08b-41b3-95c4-2b830d2d4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:40.619819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:40.768928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5272b-9346-4faa-80d8-cca58fa8b6fd None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:40.783215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5272b-9346-4faa-80d8-cca58fa8b6fd None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:40.788491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:40.789121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.789403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.789679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.798057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:40.799686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7b9e4-85b7-4829-b2c6-833afb24a6b6 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:40.808767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 729/1437] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:41.727242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b24cc-e6a3-4de0-9dcc-0edb2eb423d8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:41.727763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b24cc-e6a3-4de0-9dcc-0edb2eb423d8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:41.727880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b24cc-e6a3-4de0-9dcc-0edb2eb423d8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:41.729166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:41.734052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa49f2-8d46-46a9-b087-2ee0245d25cd None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:41.734444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa49f2-8d46-46a9-b087-2ee0245d25cd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:41.734611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa49f2-8d46-46a9-b087-2ee0245d25cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:41.752841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1afa49f2-8d46-46a9-b087-2ee0245d25cd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:41.754043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1afa49f2-8d46-46a9-b087-2ee0245d25cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:41.761840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:41.910200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b24cc-e6a3-4de0-9dcc-0edb2eb423d8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:41.924382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b24cc-e6a3-4de0-9dcc-0edb2eb423d8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:41.929624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:41.930243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:41.930524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:41.930902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:41.939447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:41.941051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bdb90-37e7-4f60-a407-2373ef8b6381 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:41.950682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 710/1440] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:44.008104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4e7fe-68d0-48c7-acf6-69845e1ddb21 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:44.008640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4e7fe-68d0-48c7-acf6-69845e1ddb21 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:44.008640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4e7fe-68d0-48c7-acf6-69845e1ddb21 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:44.010053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:44.015291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818284f1-a183-45b8-82e7-99a526ed03f1 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:44.015732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818284f1-a183-45b8-82e7-99a526ed03f1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:44.015836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818284f1-a183-45b8-82e7-99a526ed03f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:44.033875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-818284f1-a183-45b8-82e7-99a526ed03f1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:44.035092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-818284f1-a183-45b8-82e7-99a526ed03f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:44.042974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:44.081108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4e7fe-68d0-48c7-acf6-69845e1ddb21 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:44.092062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4e7fe-68d0-48c7-acf6-69845e1ddb21 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:44.095531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:44.096261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:44.107981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:44.107981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:44.107981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:44.107981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce813b3-333d-4ddd-b25e-877216789711 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:44.116592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 732/1443] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:46.536940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2e5-7189-4e3b-b84d-338262ac4055 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.537472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2e5-7189-4e3b-b84d-338262ac4055 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.537472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2e5-7189-4e3b-b84d-338262ac4055 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.538949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:46.544270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda92a66-1660-4697-aad7-7ec218b11335 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.544710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda92a66-1660-4697-aad7-7ec218b11335 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.544839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda92a66-1660-4697-aad7-7ec218b11335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.562824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fda92a66-1660-4697-aad7-7ec218b11335 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:46.564026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fda92a66-1660-4697-aad7-7ec218b11335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:46.571791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:47:46.610093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf2e5-7189-4e3b-b84d-338262ac4055 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:46.620541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf2e5-7189-4e3b-b84d-338262ac4055 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:46.624592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:46.625212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.625493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.625768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.634329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:46.635852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb213a6-614d-47c7-8ae7-304f03cd52d7 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:46.645406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 713/1446] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:49.220805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b9d0-0dab-4419-83d2-502506918b6c None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:49.221459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b9d0-0dab-4419-83d2-502506918b6c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:49.221459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b9d0-0dab-4419-83d2-502506918b6c None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:49.222766 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:49.228303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895d7d-cd90-424e-af1f-b2c09c174014 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:49.228757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895d7d-cd90-424e-af1f-b2c09c174014 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:49.228859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895d7d-cd90-424e-af1f-b2c09c174014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:49.246995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-5e895d7d-cd90-424e-af1f-b2c09c174014 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:49.248240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5e895d7d-cd90-424e-af1f-b2c09c174014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:49.256164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:49.293335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b9d0-0dab-4419-83d2-502506918b6c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:49.304328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b9d0-0dab-4419-83d2-502506918b6c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:49.307823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:49.308449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:49.308736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:49.309020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:49.317894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:49.319361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb302b-ddbf-4d76-9b3c-276679fd3f58 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:49.328908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 735/1449] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:51.647538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf3b8-8a76-42e3-912f-af8d7ccb89ca None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.648065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf3b8-8a76-42e3-912f-af8d7ccb89ca None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.648065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bf3b8-8a76-42e3-912f-af8d7ccb89ca None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.649393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:51.655377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a154c-e480-44fd-bfcf-1fa96ab977eb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.655805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a154c-e480-44fd-bfcf-1fa96ab977eb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.655945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a154c-e480-44fd-bfcf-1fa96ab977eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.673906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5a9a154c-e480-44fd-bfcf-1fa96ab977eb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:51.675135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5a9a154c-e480-44fd-bfcf-1fa96ab977eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:51.682908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:51.720941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bf3b8-8a76-42e3-912f-af8d7ccb89ca None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:51.731502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bf3b8-8a76-42e3-912f-af8d7ccb89ca None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:51.735572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:51.736190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.736475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.736754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.745673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:51.747140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e91af6-c003-4819-b7b1-33589d2ff5b7 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:51.756546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 716/1452] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:54.012081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a24d0-0d25-42ad-970f-f788852b86f4 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:54.012677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a24d0-0d25-42ad-970f-f788852b86f4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:54.012677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a24d0-0d25-42ad-970f-f788852b86f4 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:54.014625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:54.019991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a76b-4f15-42d8-b975-ad31dd642958 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:54.020446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a76b-4f15-42d8-b975-ad31dd642958 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:54.020551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a76b-4f15-42d8-b975-ad31dd642958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:54.038065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-cec3a76b-4f15-42d8-b975-ad31dd642958 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:54.039273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-cec3a76b-4f15-42d8-b975-ad31dd642958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:54.046887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:54.085061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a24d0-0d25-42ad-970f-f788852b86f4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:54.095664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a24d0-0d25-42ad-970f-f788852b86f4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:54.099307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:54.099935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:54.100219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:54.100496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:54.109931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:54.111400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae354e6d-86e9-4987-bb73-f559f4951bf8 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:54.120816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 738/1455] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:56.365546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.366066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.366172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.367576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:56.373556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eaa7c-bc43-476e-9294-553335682934 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.373980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eaa7c-bc43-476e-9294-553335682934 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.374087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eaa7c-bc43-476e-9294-553335682934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.391838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a52eaa7c-bc43-476e-9294-553335682934 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:56.393023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a52eaa7c-bc43-476e-9294-553335682934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:56.400687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:56.438084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:56.439797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:56.453327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02bcf8-d15b-4cfb-b470-523e90968a86 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:56.456783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:56.457403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.457685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.457963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.467174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:56.468753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f5918-a56b-40e8-8b12-867e4b190c9d service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:56.478008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 719/1458] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:58.744186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c0c-682f-48b7-b321-7b6454dae310 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.744722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c0c-682f-48b7-b321-7b6454dae310 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.744722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82c0c-682f-48b7-b321-7b6454dae310 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.746255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:58.751682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c520-0419-40ef-83ae-4a5a9c779ab4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.752163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c520-0419-40ef-83ae-4a5a9c779ab4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.752275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1c520-0419-40ef-83ae-4a5a9c779ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.758557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=70 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:58.770870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-67f1c520-0419-40ef-83ae-4a5a9c779ab4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:58.772071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-67f1c520-0419-40ef-83ae-4a5a9c779ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:58.780562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:58.817615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82c0c-682f-48b7-b321-7b6454dae310 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:58.823646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=73 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:58.830122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82c0c-682f-48b7-b321-7b6454dae310 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:58.833665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:58.834302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.834591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.834906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.844465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:58.845920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e439387-bb7f-4317-880c-9e0b62fdc847 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:58.855341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 741/1461] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:01.101747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b3fe-9bb7-4d5c-84b4-6af6654d42ae None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.103235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b3fe-9bb7-4d5c-84b4-6af6654d42ae None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.103235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b3fe-9bb7-4d5c-84b4-6af6654d42ae None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.103758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:01.109497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78ffb-7f2f-4c11-9002-cf7bd452811f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.109938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78ffb-7f2f-4c11-9002-cf7bd452811f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.110068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b78ffb-7f2f-4c11-9002-cf7bd452811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.128642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-28b78ffb-7f2f-4c11-9002-cf7bd452811f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:01.136818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-28b78ffb-7f2f-4c11-9002-cf7bd452811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:01.137811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:01.175276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b3fe-9bb7-4d5c-84b4-6af6654d42ae None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:01.186348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b3fe-9bb7-4d5c-84b4-6af6654d42ae None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:01.189858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:01.190490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.190806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.191086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.200587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:01.202017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde44686-1142-48c9-8256-82738c3fd7d1 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:01.211395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 722/1464] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:48:03.429390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a4874-b3c0-45c8-ae17-6ce2a35fb271 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:03.429944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a4874-b3c0-45c8-ae17-6ce2a35fb271 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:03.429944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a4874-b3c0-45c8-ae17-6ce2a35fb271 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:03.431325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:03.437054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9154-bd78-4e25-9e91-ac63de877934 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:03.437494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9154-bd78-4e25-9e91-ac63de877934 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:03.437602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9154-bd78-4e25-9e91-ac63de877934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:03.456582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6faa9154-bd78-4e25-9e91-ac63de877934 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:03.457775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6faa9154-bd78-4e25-9e91-ac63de877934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:03.465469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:03.501229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a4874-b3c0-45c8-ae17-6ce2a35fb271 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:03.511897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a4874-b3c0-45c8-ae17-6ce2a35fb271 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:03.516061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:03.516686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:03.516969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:03.517245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:03.525885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:03.527356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8962e2-54ee-41ba-8d36-8c19a617be83 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:03.536604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 744/1467] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:05.742456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed137ae-3b1a-4e6d-80c2-66c00ba87f85 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:05.743021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed137ae-3b1a-4e6d-80c2-66c00ba87f85 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:05.743021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed137ae-3b1a-4e6d-80c2-66c00ba87f85 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:05.744420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:05.750550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762c25f-0189-44c1-9ff0-22b81b62942b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:05.750971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762c25f-0189-44c1-9ff0-22b81b62942b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:05.751142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762c25f-0189-44c1-9ff0-22b81b62942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:05.769511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5762c25f-0189-44c1-9ff0-22b81b62942b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:05.770714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5762c25f-0189-44c1-9ff0-22b81b62942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:05.778303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:05.814570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed137ae-3b1a-4e6d-80c2-66c00ba87f85 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:05.826051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed137ae-3b1a-4e6d-80c2-66c00ba87f85 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:05.829600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:05.830284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:05.830656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:05.831002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:05.839885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:05.841329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba985b-9281-487a-a9d3-65395b5002d7 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:05.850452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 725/1470] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:15.765001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=17 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:15.846346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=17 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:38.916013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ad6b-5b8d-4195-9176-48c01ed238ab None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.916514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ad6b-5b8d-4195-9176-48c01ed238ab None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.916617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104ad6b-5b8d-4195-9176-48c01ed238ab None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:38.918012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 746/1471] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:38.923670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2e0e-fdf8-4739-ac86-4e942b583bf8 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.924142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2e0e-fdf8-4739-ac86-4e942b583bf8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.924238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2e0e-fdf8-4739-ac86-4e942b583bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:38.942233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f46b2e0e-fdf8-4739-ac86-4e942b583bf8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:38.943450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f46b2e0e-fdf8-4739-ac86-4e942b583bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:38.952257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 726/1472] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:38.998243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a4f6-d277-44cf-8fec-981d353ae563 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.998544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a4f6-d277-44cf-8fec-981d353ae563 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.998861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715a4f6-d277-44cf-8fec-981d353ae563 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.000229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 747/1473] 10.222.0.74 () {54 vars in 962 bytes} [Wed May 25 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:39.005739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ff1c-8d73-41dd-88c0-c4f764c02556 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:39.006014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ff1c-8d73-41dd-88c0-c4f764c02556 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:39.006298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ff1c-8d73-41dd-88c0-c4f764c02556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.024867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d301ff1c-8d73-41dd-88c0-c4f764c02556 None None] MFA Rules not processed for user `6cdeab1ae06e4df9a270a716ae0a1db3`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:39.026066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d301ff1c-8d73-41dd-88c0-c4f764c02556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:39.032439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 727/1474] 10.222.0.74 () {58 vars in 1059 bytes} [Wed May 25 14:48:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:39.039745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715a4f6-d277-44cf-8fec-981d353ae563 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:39.050701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715a4f6-d277-44cf-8fec-981d353ae563 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:39.054116 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:39.054761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:39.055041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:39.055314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.064709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:39.066477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b819c-d7e4-436b-af88-ebdefcebbbc4 service glance] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:39.076154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 748/1475] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:41.366174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa75-c474-4fe6-a4a5-c682474c95bc None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.366763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa75-c474-4fe6-a4a5-c682474c95bc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.366763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3daa75-c474-4fe6-a4a5-c682474c95bc None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:41.368260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 728/1476] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:41.373444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fd0d-1c6e-455c-820d-d6fa9b8b7e22 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.373858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fd0d-1c6e-455c-820d-d6fa9b8b7e22 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.373994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219fd0d-1c6e-455c-820d-d6fa9b8b7e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:41.392737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2219fd0d-1c6e-455c-820d-d6fa9b8b7e22 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:41.393929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2219fd0d-1c6e-455c-820d-d6fa9b8b7e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:41.402300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 749/1477] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:41.425993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3daa75-c474-4fe6-a4a5-c682474c95bc None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:41.438481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3daa75-c474-4fe6-a4a5-c682474c95bc None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:41.442899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:41.443538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.443821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.444094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:41.444817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:41.446678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:48:41.446678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:48:41.446678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:48:41.447829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ebd37-4ebb-43b4-b809-61224dcdeda2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:41.467710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 729/1478] 10.222.0.74 () {56 vars in 1108 bytes} [Wed May 25 14:48:41 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:48:43.691700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bf6c-c281-4967-9567-deb9510fc956 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.692213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bf6c-c281-4967-9567-deb9510fc956 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.692213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bf6c-c281-4967-9567-deb9510fc956 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.693550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 750/1479] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:43.699039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d263-de29-47f1-aaf6-99ba0109d142 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.699438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d263-de29-47f1-aaf6-99ba0109d142 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.699615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d263-de29-47f1-aaf6-99ba0109d142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.717374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-82c4d263-de29-47f1-aaf6-99ba0109d142 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:43.718585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-82c4d263-de29-47f1-aaf6-99ba0109d142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:43.726418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 730/1480] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:43.771806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bf6c-c281-4967-9567-deb9510fc956 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:43.786554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bf6c-c281-4967-9567-deb9510fc956 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:43.792851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:43.793648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.794004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.794372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.803335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:43.804949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efddde-183a-4c23-a92d-c481d9fb8acb service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:43.814850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 751/1481] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:46.215689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5c44-3167-46df-a044-ce1e71f012bb None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.216799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5c44-3167-46df-a044-ce1e71f012bb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.216799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5c44-3167-46df-a044-ce1e71f012bb None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.218132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 731/1482] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:46.223291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2fbf-e0f6-4d3b-b1f0-1595553902e2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.223736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2fbf-e0f6-4d3b-b1f0-1595553902e2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.223845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2fbf-e0f6-4d3b-b1f0-1595553902e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.241933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-99fc2fbf-e0f6-4d3b-b1f0-1595553902e2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:46.243168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-99fc2fbf-e0f6-4d3b-b1f0-1595553902e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:46.251010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 752/1483] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:46.286143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5c44-3167-46df-a044-ce1e71f012bb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:46.297233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5c44-3167-46df-a044-ce1e71f012bb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:46.300744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:46.301374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.301657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.301933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.311073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:46.312501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b18461-dd2c-47e7-a199-717a693d9c50 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:46.321948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 732/1484] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:48.565618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495915-7764-4bb4-b540-e12c86a744fa None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:48.566235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495915-7764-4bb4-b540-e12c86a744fa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:48.566235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495915-7764-4bb4-b540-e12c86a744fa None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:48.567602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 753/1485] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:48.573114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781272f3-39de-4a8c-bf4d-b2c14683ee45 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:48.573801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781272f3-39de-4a8c-bf4d-b2c14683ee45 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:48.574038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781272f3-39de-4a8c-bf4d-b2c14683ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:48.592502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-781272f3-39de-4a8c-bf4d-b2c14683ee45 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:48.593630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-781272f3-39de-4a8c-bf4d-b2c14683ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:48.601380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 733/1486] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:48.638915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f495915-7764-4bb4-b540-e12c86a744fa None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:48.649485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f495915-7764-4bb4-b540-e12c86a744fa None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:48.653045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:48.653674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:48.653960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:48.654241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:48.663095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:48.664532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb740b6-99ae-4750-8334-19fab1a97e8a service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:48.673749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 754/1487] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:51.029225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b13c-4edb-4ca7-b064-082842d9c470 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.029890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b13c-4edb-4ca7-b064-082842d9c470 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.029890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b13c-4edb-4ca7-b064-082842d9c470 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.031194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 734/1488] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:51.036467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a374e-20fa-4cc4-905e-2ef0acc6011f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.036921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a374e-20fa-4cc4-905e-2ef0acc6011f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.037027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a374e-20fa-4cc4-905e-2ef0acc6011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.054964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ac4a374e-20fa-4cc4-905e-2ef0acc6011f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:51.056139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ac4a374e-20fa-4cc4-905e-2ef0acc6011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:51.064374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 755/1489] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:51.111050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4b13c-4edb-4ca7-b064-082842d9c470 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:51.114598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4b13c-4edb-4ca7-b064-082842d9c470 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:51.118114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:51.118808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.119097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.119376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.128911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:51.130334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa57ce8-367d-46d7-b3bb-369103e84851 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:51.139728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 735/1490] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:53.714898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47854-7fd5-4e06-9335-ea90ecac7b56 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:53.715483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47854-7fd5-4e06-9335-ea90ecac7b56 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:53.715483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d47854-7fd5-4e06-9335-ea90ecac7b56 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:53.717291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 756/1491] 10.222.0.74 () {54 vars in 926 bytes} [Wed May 25 14:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:48:53.722689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75c33-6f20-4d62-8586-4ec4f8dff23f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:53.723149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75c33-6f20-4d62-8586-4ec4f8dff23f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:53.723283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75c33-6f20-4d62-8586-4ec4f8dff23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:53.741346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c6b75c33-6f20-4d62-8586-4ec4f8dff23f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:53.742544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c6b75c33-6f20-4d62-8586-4ec4f8dff23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:53.750289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 736/1492] 10.222.0.74 () {58 vars in 1023 bytes} [Wed May 25 14:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:53.789127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d47854-7fd5-4e06-9335-ea90ecac7b56 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:53.800259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d47854-7fd5-4e06-9335-ea90ecac7b56 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:53.803768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:53.804396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:53.804678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:53.804955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:53.813733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:53.815200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11402fd-9473-4064-9bc7-7f1259146308 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:53.824685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 757/1493] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:03.736292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=33 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:03.820302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=33 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:42.065499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.066115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.066115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.084286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:50:42.085500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:42.093454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 737/1494] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:50:42.102875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.103410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.103541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.122476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:50:42.123824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:42.132185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 758/1495] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:50:42.137029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.146727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bf6f-7e64-4c54-a4f5-39dbe2add05e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.151254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.151901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.152193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.152472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.153195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:50:42.153539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.154552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.154552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:50:42.154552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:50:42.155524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.167231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 738/1496] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:50:42.175953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.185782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b2d96-0241-4262-aaf8-89ca5edbfed4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.190411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.191088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.191375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.191656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.192352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.194099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.194099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:50:42.194099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:50:42.195323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.310192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 759/1497] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:50:42.314707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.325176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa5f4-78a3-4311-b449-d7f4e4ffe58c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.383062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.383819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.384157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.384489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.385230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.386348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.386348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:50:42.386348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:50:42.387895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.489286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 739/1498] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:50:42.493711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.504045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5ebd8-d59a-4396-adf6-2d4500d96075 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.508817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.509477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.509824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.510107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.510907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.511962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.511962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:50:42.511962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:50:42.512842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.525693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 760/1499] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:50:42.530502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.540619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd188ec-16d0-4fcb-9a6e-3cac4478c404 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.545939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.546599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.546917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.547202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.547929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.548943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.548943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:50:42.548943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:50:42.549835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6966f-2764-4f83-8eec-215c04940cf0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.563570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 740/1500] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:50:42.568244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.579282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099d33a-a712-4a88-bab9-dfc59138b177 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.583769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.584645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.584930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.585213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] PATH_INFO: `/v3/projects/8da626068bcf4b4bb4b24821b63402b8/users/8effb9190e2543829a04865f36c52133/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.601221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da626068bcf4b4bb4b24821b63402b8, user_id=8effb9190e2543829a04865f36c52133, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.602274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:42.602274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:50:42.602274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:50:42.604276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c21094-2ec7-4bc7-b8e5-2d4e22a5f4f0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.702446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 761/1501] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:50:42 2022] PUT /identity/v3/projects/8da626068bcf4b4bb4b24821b63402b8/users/8effb9190e2543829a04865f36c52133/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:50:42.713793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec8673-16f4-4739-82d8-3c4abb742cdd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.714090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec8673-16f4-4739-82d8-3c4abb742cdd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.714376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec8673-16f4-4739-82d8-3c4abb742cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.741651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-5fec8673-16f4-4739-82d8-3c4abb742cdd None None] MFA Rules not processed for user `8effb9190e2543829a04865f36c52133`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:50:42.742898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5fec8673-16f4-4739-82d8-3c4abb742cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:42.808313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 741/1502] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:50:42.825906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e36e-83b9-4951-aeac-1f87f14c4bf0 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.826275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e36e-83b9-4951-aeac-1f87f14c4bf0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.830829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627e36e-83b9-4951-aeac-1f87f14c4bf0 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.830829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 762/1503] 10.222.0.74 () {54 vars in 963 bytes} [Wed May 25 14:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:50:42.833165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc66da-e588-4150-a350-b31a87f2a9c5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.833543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc66da-e588-4150-a350-b31a87f2a9c5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.833866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc66da-e588-4150-a350-b31a87f2a9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.854523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-05dc66da-e588-4150-a350-b31a87f2a9c5 None None] MFA Rules not processed for user `ffea1035a1d2434e948256087a112b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:50:42.855786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-05dc66da-e588-4150-a350-b31a87f2a9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:42.901662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:42.909811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 742/1504] 10.222.0.74 () {58 vars in 1060 bytes} [Wed May 25 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:50:42.917871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627e36e-83b9-4951-aeac-1f87f14c4bf0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:42.928845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627e36e-83b9-4951-aeac-1f87f14c4bf0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:42.932525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:42.933161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.933443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.933722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:42.942453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:42.943928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:42.953582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 763/1505] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:50:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:50:42.998197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b28c-8252-4e62-ad7d-7c71b9fc756e None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:42.998491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b28c-8252-4e62-ad7d-7c71b9fc756e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:42.998800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b28c-8252-4e62-ad7d-7c71b9fc756e None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:43.000138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 743/1506] 10.222.0.74 () {52 vars in 829 bytes} [Wed May 25 14:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:50:43.004004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:43.065643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b6cd7-a30c-4aa3-8df5-9e4959c33da9 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:43.070255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:43.071182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:43.071725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:43.071998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] PATH_INFO: `/v3/users/8effb9190e2543829a04865f36c52133` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:43.073212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8effb9190e2543829a04865f36c52133)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:43.074241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:43.074241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:50:43.074241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:50:43.075997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:43.451761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8effb9190e2543829a04865f36c52133'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:43.464400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8effb9190e2543829a04865f36c52133'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:43.465165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8effb9190e2543829a04865f36c52133'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:43.483508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8effb9190e2543829a04865f36c52133'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:43.487462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:43.562935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8effb9190e2543829a04865f36c52133'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:43.569429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 764/1507] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:50:43 2022] DELETE /identity/v3/users/8effb9190e2543829a04865f36c52133 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:50:43.574515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:43.575059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:43.575193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:43.592782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:50:43.593964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:43.655541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 744/1508] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:50:43.666724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:43.668138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:43.725139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72c839-8965-4e9e-a54e-74a12e4eae85 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:43.728851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:43.729499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:43.729811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:43.730070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:43.739657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:43.741215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34965fdd-2bb0-4d09-a207-b8a1a5837a19 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:43.751291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 765/1509] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:50:44.195657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:44.195657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:44.209026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0eeb5f-8bd7-4dda-89ab-b43cf50dcef0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:44.216224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:44.217827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:44.218177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:44.218701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] PATH_INFO: `/v3/projects/8da626068bcf4b4bb4b24821b63402b8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:44.220888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da626068bcf4b4bb4b24821b63402b8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:44.222755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:50:44.222755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:50:44.222755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:50:44.224680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:44.341462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da626068bcf4b4bb4b24821b63402b8', 'request_id': 'req-2c632573-4db0-41f6-9e1d-9ebed86d5248'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:44.416212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da626068bcf4b4bb4b24821b63402b8', 'request_id': 'req-2c632573-4db0-41f6-9e1d-9ebed86d5248'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:44.430546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da626068bcf4b4bb4b24821b63402b8', 'request_id': 'req-2c632573-4db0-41f6-9e1d-9ebed86d5248'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:44.441750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] The token cache is being invalidate because project 8da626068bcf4b4bb4b24821b63402b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:50:44.441981 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:44.442774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2c632573-4db0-41f6-9e1d-9ebed86d5248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:50:44.444330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 745/1510] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:50:44 2022] DELETE /identity/v3/projects/8da626068bcf4b4bb4b24821b63402b8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:50:53.746992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:54.438557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:28.577010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:28.577593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:28.577593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:28.579020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:28.584686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfa45-0f1a-4d11-a8de-7f52297af01a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:28.585142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfa45-0f1a-4d11-a8de-7f52297af01a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:28.585274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dfa45-0f1a-4d11-a8de-7f52297af01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:28.603193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-5f9dfa45-0f1a-4d11-a8de-7f52297af01a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:28.604395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5f9dfa45-0f1a-4d11-a8de-7f52297af01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:28.662067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:28.832524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:28.834423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:28.914020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33b547-454d-4baf-8f57-70205108d37f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:28.919703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:28.920376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:28.920668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:28.920949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:28.930143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:28.931866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f636a-aae2-4eb6-bdd4-428540eb2b84 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:28.942042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 767/1513] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:29.821459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00213a-17ca-4054-aa68-b7fc2401132d None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:29.822304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00213a-17ca-4054-aa68-b7fc2401132d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:29.822510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00213a-17ca-4054-aa68-b7fc2401132d None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:29.824029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:29.829695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76f65-1ba4-407a-b16f-f0b5299f73f0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:29.830160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76f65-1ba4-407a-b16f-f0b5299f73f0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:29.830269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76f65-1ba4-407a-b16f-f0b5299f73f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:29.848192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-85b76f65-1ba4-407a-b16f-f0b5299f73f0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:29.849416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-85b76f65-1ba4-407a-b16f-f0b5299f73f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:29.857355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:30.014216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00213a-17ca-4054-aa68-b7fc2401132d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:30.028663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00213a-17ca-4054-aa68-b7fc2401132d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:30.034524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:30.035195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:30.035483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:30.035762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:30.044227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:30.045839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b460da-a393-407a-a990-d06294d7c57c service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:30.056337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 748/1516] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:52:30.651229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33917e-d44c-4c83-953f-442ce85829d3 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:30.651485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33917e-d44c-4c83-953f-442ce85829d3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:30.651706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33917e-d44c-4c83-953f-442ce85829d3 None None] PATH_INFO: `/v3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:30.653005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 769/1517] 10.222.0.86 () {54 vars in 925 bytes} [Wed May 25 14:52:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:52:30.661060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3c1b-4da1-46f0-b9e4-dfa67f7963ae None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:30.661400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3c1b-4da1-46f0-b9e4-dfa67f7963ae None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:30.661722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3c1b-4da1-46f0-b9e4-dfa67f7963ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:30.680821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-858b3c1b-4da1-46f0-b9e4-dfa67f7963ae None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:30.682045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-858b3c1b-4da1-46f0-b9e4-dfa67f7963ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:30.744847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 749/1518] 10.222.0.86 () {58 vars in 1015 bytes} [Wed May 25 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:39.842865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=37 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:40.735740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=62 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:41.176781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0239-8bfc-4508-857f-92b391010599 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.177615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0239-8bfc-4508-857f-92b391010599 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.177780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0239-8bfc-4508-857f-92b391010599 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.179446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 770/1519] 10.222.0.74 () {54 vars in 960 bytes} [Wed May 25 14:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:41.184561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72015be-2c9b-4aed-891b-72530cc43d84 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.184835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72015be-2c9b-4aed-891b-72530cc43d84 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.185094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72015be-2c9b-4aed-891b-72530cc43d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.204058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e72015be-2c9b-4aed-891b-72530cc43d84 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:41.205285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e72015be-2c9b-4aed-891b-72530cc43d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:41.213930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 750/1520] 10.222.0.74 () {58 vars in 1057 bytes} [Wed May 25 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:41.222690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0239-8bfc-4508-857f-92b391010599 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:41.237724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0239-8bfc-4508-857f-92b391010599 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:41.242833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:41.243481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.243774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.244048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.252949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:41.254574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aa353-e598-46c4-926b-bc7b303515c7 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:41.264621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 771/1521] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:41.330907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010224c-821a-4ebb-ad5a-e29ee2c52e80 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.332680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010224c-821a-4ebb-ad5a-e29ee2c52e80 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.332879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010224c-821a-4ebb-ad5a-e29ee2c52e80 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.345074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:52:41 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:41.350885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58e409-81af-4a75-b8a6-ccc85bc4d53b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.351378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58e409-81af-4a75-b8a6-ccc85bc4d53b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.351845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58e409-81af-4a75-b8a6-ccc85bc4d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.372019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4c58e409-81af-4a75-b8a6-ccc85bc4d53b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:41.373231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4c58e409-81af-4a75-b8a6-ccc85bc4d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:41.381542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:41.541327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010224c-821a-4ebb-ad5a-e29ee2c52e80 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:41.556996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010224c-821a-4ebb-ad5a-e29ee2c52e80 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:41.564177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:41.564811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:41.565091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:41.565363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:41.575196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:41.582443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60a685-75ea-4d6f-b375-0339f461cb63 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:41.588098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 752/1524] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:42.511411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d9ff-3679-4115-86ad-252e25994571 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:42.512029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d9ff-3679-4115-86ad-252e25994571 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:42.512029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d9ff-3679-4115-86ad-252e25994571 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:42.513353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:42.518868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e39d3-13fe-4b29-9636-b8a3f5d029b9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:42.519329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e39d3-13fe-4b29-9636-b8a3f5d029b9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:42.519523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e39d3-13fe-4b29-9636-b8a3f5d029b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:42.539920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0b2e39d3-13fe-4b29-9636-b8a3f5d029b9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:42.541033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0b2e39d3-13fe-4b29-9636-b8a3f5d029b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:42.548738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:42.700292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d9ff-3679-4115-86ad-252e25994571 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:42.715347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d9ff-3679-4115-86ad-252e25994571 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:42.720793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:42.721424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:42.721710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:42.721990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:42.730038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:42.731654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca959f-aa1c-4561-8a3c-5bd2bb7423cf service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:42.740960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 774/1527] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:47.752226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:47.752894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:47.752894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:47.754158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:47.759495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2d6-19b5-446d-8a15-b4f006faff0e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:47.759901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2d6-19b5-446d-8a15-b4f006faff0e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:47.760060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1d2d6-19b5-446d-8a15-b4f006faff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:47.778311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d1d1d2d6-19b5-446d-8a15-b4f006faff0e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:47.779550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d1d1d2d6-19b5-446d-8a15-b4f006faff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:47.788155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:47.826207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:47.828207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:48.025632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee184-ffd4-439d-8189-baf014e4ef6f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:48.029172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:48.029818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:48.030100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:48.030375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:48.040153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:48.041680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d49bc-f9bd-4748-a285-82e8a4d7bed0 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:48.052024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 755/1530] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:50.251980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2d8b-9729-4432-b4d8-dfe39e612175 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:50.252523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2d8b-9729-4432-b4d8-dfe39e612175 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:50.252523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada2d8b-9729-4432-b4d8-dfe39e612175 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:50.253882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:50.259346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723e53d-970d-442f-90e8-148c080f06ed None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:50.259782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723e53d-970d-442f-90e8-148c080f06ed None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:50.259908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723e53d-970d-442f-90e8-148c080f06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:50.278333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c723e53d-970d-442f-90e8-148c080f06ed None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:50.279576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c723e53d-970d-442f-90e8-148c080f06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:50.287439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:50.323041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada2d8b-9729-4432-b4d8-dfe39e612175 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:50.334439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada2d8b-9729-4432-b4d8-dfe39e612175 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:50.337948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:50.338575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:50.338911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:50.339185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:50.347870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:50.349299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0b93b-8eb7-4989-b479-f8cbfd1c8ac2 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:50.360051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 777/1533] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:54.158592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:54.159193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:54.159193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:54.160558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:54.165854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a16f-a92d-481e-a2f0-c60532f3f280 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:54.166287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a16f-a92d-481e-a2f0-c60532f3f280 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:54.166402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2a16f-a92d-481e-a2f0-c60532f3f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:54.185955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-36e2a16f-a92d-481e-a2f0-c60532f3f280 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:54.187177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-36e2a16f-a92d-481e-a2f0-c60532f3f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:54.195632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:54.466892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:54.468576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:54.525814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04064af7-9f8a-418c-b250-8c8c8a1892f4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:54.529338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:54.529978 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:54.530263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:54.530543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:54.539425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:54.540847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75af647-a7b9-424c-a713-980764145537 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:54.550666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 758/1536] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:56.773880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:56.774533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:56.774533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:56.775848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:52:56.781124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506fe9-adf8-433f-bc6b-59e9d55a81b5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:56.782117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506fe9-adf8-433f-bc6b-59e9d55a81b5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:56.782339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb506fe9-adf8-433f-bc6b-59e9d55a81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:56.800485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-eb506fe9-adf8-433f-bc6b-59e9d55a81b5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:52:56.801714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-eb506fe9-adf8-433f-bc6b-59e9d55a81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:56.809416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:52:57.084354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:57.086046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:52:57.100296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21931c9-1ab3-40e0-a414-8ecd04201a64 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:57.103795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:57.104425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:57.104712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:57.104990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:57.114511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:57.116145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29ad9-8950-4f4b-8992-950a8ab15f74 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:57.125453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 780/1539] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:53:06.795683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=70 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:53:07.121359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=33 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:30.072322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.072917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.072917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.100067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.101674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.114302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 760/1540] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.124322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bef172-bd0f-4add-9989-d18414685b7b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.125715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bef172-bd0f-4add-9989-d18414685b7b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.125715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bef172-bd0f-4add-9989-d18414685b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.160906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-24bef172-bd0f-4add-9989-d18414685b7b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.171784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-24bef172-bd0f-4add-9989-d18414685b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.176807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 781/1541] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.181098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.182317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adbd78-22a0-484c-91fe-485a8108fb98 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.182613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adbd78-22a0-484c-91fe-485a8108fb98 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.182947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adbd78-22a0-484c-91fe-485a8108fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.194166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbe1ca-a7f7-4d16-8491-2b6f41d785bf None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.208188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.210076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.210628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.211162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.212135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:30.212685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.214613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.216669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-75adbd78-22a0-484c-91fe-485a8108fb98 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.216928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-75adbd78-22a0-484c-91fe-485a8108fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.242817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 761/1542] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:30.253234 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 782/1543] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.256140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.265817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.266131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.266403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.269532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39aae-8a92-494b-8a80-aec560f52560 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.280590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.281667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.282163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.282703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.283653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.285164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:30.285164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:54:30.285164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:54:30.286882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.307950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.309276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.338399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 783/1544] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.343320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.354531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b5a4d-40f2-4c29-b30a-225ab3d7262f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.364061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.364757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.365043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.365322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.366093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:30.366412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.367606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:30.367606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:54:30.367606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:54:30.368551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.390033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 784/1545] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:30.398337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.408848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736880d-b441-4bda-b7ca-36ffda09a311 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.419317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.420061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.420271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.422194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.422939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.425469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936b6ae-1ef6-4e47-9782-2befe4bc2b80 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.441387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 762/1546] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:30.450542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.553267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 785/1547] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/projects => generated 405 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:54:30.555789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.557347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.560283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664d18b-05c8-4b26-b937-414b63eb29f5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.560790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.598824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.601415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.669476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.670264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.670611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.670920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.671817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.674011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f301a8-d8c7-46b7-8900-df19eeea5ecd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.691251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 786/1548] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.692401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.701828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7959a-e5a8-49d9-ae36-1335d334cf14 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.707283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.707937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.708222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.708502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.709384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.710462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:30.710462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:54:30.710462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:54:30.711679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f320f2-f35a-4ac8-bc87-af0d8972e3c9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.747829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 763/1549] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:54:30.749462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5b91f-82a6-4a37-a0ec-84231c1b58e4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.749642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5b91f-82a6-4a37-a0ec-84231c1b58e4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.749908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5b91f-82a6-4a37-a0ec-84231c1b58e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.772834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-90f5b91f-82a6-4a37-a0ec-84231c1b58e4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.774286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-90f5b91f-82a6-4a37-a0ec-84231c1b58e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.783736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 764/1550] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.785193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73d120-0894-467d-903f-287fc1b20b96 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.785479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73d120-0894-467d-903f-287fc1b20b96 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.785793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73d120-0894-467d-903f-287fc1b20b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.793838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 787/1551] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:30.795155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a14d-b364-427d-b7f1-1f501461df94 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.795452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a14d-b364-427d-b7f1-1f501461df94 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.795738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a14d-b364-427d-b7f1-1f501461df94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.805019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6c73d120-0894-467d-903f-287fc1b20b96 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.806228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6c73d120-0894-467d-903f-287fc1b20b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.814499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-bdb7a14d-b364-427d-b7f1-1f501461df94 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.814884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 765/1552] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.815826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bdb7a14d-b364-427d-b7f1-1f501461df94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.816122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26ea-a04c-4ce4-8efe-fb38daf48ea7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.816410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26ea-a04c-4ce4-8efe-fb38daf48ea7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.816694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab26ea-a04c-4ce4-8efe-fb38daf48ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.824706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 788/1553] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.826075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.826369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.826665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.839573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-66ab26ea-a04c-4ce4-8efe-fb38daf48ea7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.840820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-66ab26ea-a04c-4ce4-8efe-fb38daf48ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.845505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.846675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.852504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 766/1554] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.852504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae606f2-5d46-49a7-8164-827dd5c27ada None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.852504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae606f2-5d46-49a7-8164-827dd5c27ada None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.852504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae606f2-5d46-49a7-8164-827dd5c27ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.861422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 789/1555] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.862316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.874239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accc87-a06a-4a26-9522-e59aa530fb34 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.877403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2ae606f2-5d46-49a7-8164-827dd5c27ada None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.878607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2ae606f2-5d46-49a7-8164-827dd5c27ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.879588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.880247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.880648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.881045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.881779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.883447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.887102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 767/1556] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.888663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82734-a43b-4690-a5d1-603d2b942a4a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.888942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82734-a43b-4690-a5d1-603d2b942a4a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.889220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82734-a43b-4690-a5d1-603d2b942a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.893937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 790/1557] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:30.894653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.904467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de725b-0864-43a8-b581-85b7369140be admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.909128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.909775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.910062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.910340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.911086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.911221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-43c82734-a43b-4690-a5d1-603d2b942a4a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.912298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-43c82734-a43b-4690-a5d1-603d2b942a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.912825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e1c26-d1f5-4fc2-9f30-8522c7b38a56 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.920081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 768/1558] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.921447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc49f-4a9d-4c49-a249-74043ba67c03 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.921732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc49f-4a9d-4c49-a249-74043ba67c03 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.922005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc49f-4a9d-4c49-a249-74043ba67c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.926510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 791/1559] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:30.927796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.928089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.928367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.943253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c96cc49f-4a9d-4c49-a249-74043ba67c03 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.944426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c96cc49f-4a9d-4c49-a249-74043ba67c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.946796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.947960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.952245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 769/1560] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.955952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f903d9-5d30-479b-b533-3214181bbd81 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.956250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f903d9-5d30-479b-b533-3214181bbd81 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.956573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f903d9-5d30-479b-b533-3214181bbd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.957857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 792/1561] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.957857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:30.968061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb34db7-e656-4fac-b2ba-5fb20c943a9b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:30.972684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:30.973324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.973611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.973889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.974399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a7f903d9-5d30-479b-b533-3214181bbd81 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:30.974606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:30.974954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:30.975622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a7f903d9-5d30-479b-b533-3214181bbd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:30.976585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:30.985512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 770/1562] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:30.988225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:30.988510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:30.990602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 793/1563] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:30.990808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:30.990936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.001026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3a70-2bdf-4e5a-86d7-49259b9a887f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.005704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.006348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.006648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.006943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.007173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:31.007701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.008354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:31.009313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.016112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 771/1564] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:31.017424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.023058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 794/1565] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:31.023058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.023315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:31.028604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a569ec2-f760-4ead-943d-fd4cb9545f96 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.033295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e868c-ef3f-4fae-8391-d77c53c935d1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.033915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.034566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.034883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.035161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.035885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:31.036193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.037826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.037943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.038562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.038886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.039159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.039875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.041642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.050389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 772/1566] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:31.051321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.051825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 795/1567] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:31.052589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.064426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c556c-6c35-4576-a71f-8be7e21429c5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.066184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748f1a3-b098-4978-a060-1c21ad6eb1f1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.069402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.070173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.070441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.070747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.071150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.071515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:31.071796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.071898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.072106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.072388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.073111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:31.073393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.073493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.075202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.081192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:31.086405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 796/1568] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:31.087307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.088097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 773/1569] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:31.088681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.097505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e674d-fb30-4a5b-8ec6-bba95d5f3cd3 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.098575 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7f376-78ac-4f54-8df6-00b9245c5dab admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.102901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.103566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.103854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.103968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.104276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.104502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.104788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.105012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:31.105127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.105366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.105858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.107088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.107792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.127856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 797/1570] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:31.127856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.131277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d480e-11ba-473b-b958-8bee498e4c9a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.136451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.137120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.137405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.137678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.138438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:31.138782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.140413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.152336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 798/1571] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:31.156379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.162605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9d80-78f8-411d-8898-b8865b37b228 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.167164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.168012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.168289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.168564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] PATH_INFO: `/v3/projects/c86222a9c32a4f77878dc82c40e38ff1/users/cd25c0ad8e1c4fa08eca9b6bddeca55d/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.183730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 774/1572] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.186204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.189516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86222a9c32a4f77878dc82c40e38ff1, user_id=cd25c0ad8e1c4fa08eca9b6bddeca55d, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.195395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.249217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694993-cd38-44a7-b444-2809237c53be admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.253964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.254879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.255161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.255471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] PATH_INFO: `/v3/projects/66c7e071d94c432ba5b9b1ca57c0cafd/users/9eddf57ad0324f8dbde26150934df50a/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.269660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c7e071d94c432ba5b9b1ca57c0cafd, user_id=9eddf57ad0324f8dbde26150934df50a, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.274768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:31.274768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:54:31.274768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:54:31.274768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.277415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 799/1573] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:31 2022] PUT /identity/v3/projects/c86222a9c32a4f77878dc82c40e38ff1/users/cd25c0ad8e1c4fa08eca9b6bddeca55d/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:31.278235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.342569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5baa-319c-4e8c-b303-82b86a00bbc5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.348053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.348713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.348999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.349323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.350027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.352002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.369349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 775/1574] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:31 2022] PUT /identity/v3/projects/66c7e071d94c432ba5b9b1ca57c0cafd/users/9eddf57ad0324f8dbde26150934df50a/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:31.369349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.436007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d922fa6-71c7-49fb-bde2-8555e8d75748 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.441680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.442946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.443369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.443730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.444585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.444688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 800/1575] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.445569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.450712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.507707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822016d0-0159-4e8c-b920-f5d07cb73c5a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.512488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.513148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.518482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.518482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.518482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.518482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.541750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 776/1576] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.556342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.568067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:31.605818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0ed8-1568-4ea7-89a6-c617417aee6f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.610471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.611680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.616454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.616454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.616454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.616454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.638761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 801/1577] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.639568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.657933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:31.700219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8622cf-10f2-4ac5-9b65-aa5b6e6b0334 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.704994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.705662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.705947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.706229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.706956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.708865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764f340-7ec5-484f-953a-70d6af3fd7e3 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.723837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 777/1578] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.728234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.791140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a2488-6fb9-4167-8c31-64570b9a5549 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.795816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.796477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.796765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.797042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.797731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.800211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.822230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 802/1579] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.824211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4a29-c77c-4fbc-aabd-71c5fc32b1d3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.824517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4a29-c77c-4fbc-aabd-71c5fc32b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.824798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa4a29-c77c-4fbc-aabd-71c5fc32b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.841932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-46aa4a29-c77c-4fbc-aabd-71c5fc32b1d3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:31.843165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-46aa4a29-c77c-4fbc-aabd-71c5fc32b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:31.900198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 803/1580] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:31.901543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.901837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.902116 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.914793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 778/1581] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:31.915675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.919286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:31.920926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:31.926178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739a7dd-cf36-4ace-9d0a-909c85ecc33c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.929457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 804/1582] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:31.930187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:31.933622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.934295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.934583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.934894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.935592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.937631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:31.941120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443258d-ab33-4b4f-ab69-8613e77d033a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:31.946349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:31.947085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:31.947375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:31.947656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:31.948343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:31.950335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.037712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 779/1583] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:32.038625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.048857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f11ca7-327b-40c8-a58f-d6902fd5fb6c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.053404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.054032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.054318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.054602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.055309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.057210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.066701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 805/1584] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:31 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:32.067751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.078952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16b965-f381-4c7b-a9bf-a2fcb97c2433 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.084462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.085198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.086301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.086301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.086798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.088942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.136318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 780/1585] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:32 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:32.137141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.146841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a2a0-b40c-4850-9b98-70b00e43b8eb admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.151564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.152127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.152409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.152685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.153360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.155395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.164930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 806/1586] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:32 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:32.168735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.225260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97d7-7faa-401d-a387-eec0d0bd3a37 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.228955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.229596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.229882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.230185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.239946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.241408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.250971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 807/1587] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:32.251757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.257465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:32.261127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124d34f-a55a-4b2b-ba71-9ad34a7ea546 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.266136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.267154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.267273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.267708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.268430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.270269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.283728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 808/1588] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.287613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 781/1589] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:32 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:32.287740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.287872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.296013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b776fb-275b-417e-bcd6-e68471885670 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.297385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9a54e-0f7c-4602-aa4d-1905ed7ad527 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.301617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.301903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.302456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.302707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.302880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.302982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.303080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.303338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.304129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.305791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.312211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.313676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.320567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 782/1590] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.324909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.325045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 809/1591] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:32.325045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.336197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a61fe-7cd7-457d-9ca4-47472069a320 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.341327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f096ebd-9067-4000-b8fc-a00ad9c566fe admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.341462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.342038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.342328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.342614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.343376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.345531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.348597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.349284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.349549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.349963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.350692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.352533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.357604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 810/1592] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.358671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.365204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 783/1593] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.365976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.372020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459f44e-18df-41c7-be52-b21a23a9d57e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.375863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01478ed4-7eef-44c4-b6a2-891649deb6ab admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.376999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.377671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.377957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.378406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.380644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.381288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.381566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.381838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.382553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.383659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.388229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.388365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.398747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 784/1594] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.398747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.398991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 811/1595] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.398991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.408160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b63b1-1f6b-4da7-9677-8c41de9c4912 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.411318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb5c0-1049-43ba-9412-f3021071a1e0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.412975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.413948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.414304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.414720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.415465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.417135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.417699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.418444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.418964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.419307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.420576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.422528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.423539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:32.429055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 785/1596] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.432736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.435602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 812/1597] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.437557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.440676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755197-8520-4398-a951-88b29d332daa admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.445902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.446572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.446885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.447166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.447890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.449838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.450383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d367d-896b-4039-bb46-e2aed7d13d0b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.456474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.461497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.461497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.461497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.461497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.461497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.468699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 786/1598] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.468699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.472207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 813/1599] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.473221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.484670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ffd51-bfa1-41b2-8119-d2dd2ec21f0a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.489871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.490146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9fbf7-62af-4ff0-b308-84622127d8f5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.490548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.490887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.491168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.491892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.493492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.498343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.500247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.500656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.501839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.503003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.505273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.506144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 814/1600] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.510004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.521796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083ff42-5fc1-41be-b7a9-0c078976b1b4 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.522053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 787/1601] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:32.523292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.527246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.528233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.528593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.528988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] PATH_INFO: `/v3/projects/9263e6d657dd458d9f497bb868adb941/users/becb6feb52b643279066aef1a7a9f063/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.536771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0268b-1379-45b0-bd82-80a42d85bb65 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.542899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.547105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.547105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.547105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] PATH_INFO: `/v3/projects/18c8213736cf4113951fbb3520c18183/users/237510469a8e4e36ad40a34cdf705656/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.547418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9263e6d657dd458d9f497bb868adb941, user_id=becb6feb52b643279066aef1a7a9f063, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.550021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.566024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c8213736cf4113951fbb3520c18183, user_id=237510469a8e4e36ad40a34cdf705656, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.570082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.605508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 815/1602] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/9263e6d657dd458d9f497bb868adb941/users/becb6feb52b643279066aef1a7a9f063/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:32.609780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.653381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 788/1603] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/18c8213736cf4113951fbb3520c18183/users/237510469a8e4e36ad40a34cdf705656/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:32.653381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.687320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc560092-54f1-4411-80b5-8268bbedb7ac admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.737103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee51bc-7251-4339-a3ab-9202ef50d0c4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:32.753129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.753129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.753129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.753129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] PATH_INFO: `/v3/projects/89a36e186c5f4d3c8b9030d91f6a30f6/users/9dba831ebb584de5a0c84f39901d3c32/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.762547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a36e186c5f4d3c8b9030d91f6a30f6, user_id=9dba831ebb584de5a0c84f39901d3c32, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.765353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.766215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:32.767809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:32.768172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:32.768520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] PATH_INFO: `/v3/projects/79333a7f533640e99bd21b9c8be032f7/users/79c1fa4f64dc42d9b2b3e3e7ac363343/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:32.780872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:32.784264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=79333a7f533640e99bd21b9c8be032f7, user_id=79c1fa4f64dc42d9b2b3e3e7ac363343, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:32.787167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:32.843708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 789/1604] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/89a36e186c5f4d3c8b9030d91f6a30f6/users/9dba831ebb584de5a0c84f39901d3c32/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:32.844538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.897377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 816/1605] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/79333a7f533640e99bd21b9c8be032f7/users/79c1fa4f64dc42d9b2b3e3e7ac363343/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:32.898134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:32.918243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461677-02b1-42e5-ba44-948077dc92ed admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.015384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:33.032389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.033316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.033601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.033885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] PATH_INFO: `/v3/projects/4a1a1173d9a24de2b797fdf1c72f4131/users/8a96ba1833514a7ba5f17409d0781026/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.038713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f8b7d-92e2-44a0-ae54-633b7b51f4db admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.044803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.045985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.046477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.046941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] PATH_INFO: `/v3/projects/23ff6c12f7604fe589925cd62611ccc3/users/2e20671c2bde4268b88a66924290017a/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.052057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1a1173d9a24de2b797fdf1c72f4131, user_id=8a96ba1833514a7ba5f17409d0781026, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.054992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d8d64-4690-40f3-8b63-8445ab0805b0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.064921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ff6c12f7604fe589925cd62611ccc3, user_id=2e20671c2bde4268b88a66924290017a, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.067980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6082746-b0f4-484e-9d73-67ec8e5590ac admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.157291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 790/1606] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/4a1a1173d9a24de2b797fdf1c72f4131/users/8a96ba1833514a7ba5f17409d0781026/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:33.159313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef79b-42f0-4b4f-b817-e4152d4476f0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.159611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef79b-42f0-4b4f-b817-e4152d4476f0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.159890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef79b-42f0-4b4f-b817-e4152d4476f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.160392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 817/1607] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:32 2022] PUT /identity/v3/projects/23ff6c12f7604fe589925cd62611ccc3/users/2e20671c2bde4268b88a66924290017a/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:33.164354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3032-88e9-4072-af5f-098c3a84bb86 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.164641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3032-88e9-4072-af5f-098c3a84bb86 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.164918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3032-88e9-4072-af5f-098c3a84bb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.189120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b09ef79b-42f0-4b4f-b817-e4152d4476f0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.190303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b09ef79b-42f0-4b4f-b817-e4152d4476f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.191702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-67db3032-88e9-4072-af5f-098c3a84bb86 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.193150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-67db3032-88e9-4072-af5f-098c3a84bb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.282041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 791/1608] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.284639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9e36a-b0f0-4ec6-91ce-c2960a253e9c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.285047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9e36a-b0f0-4ec6-91ce-c2960a253e9c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.285428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9e36a-b0f0-4ec6-91ce-c2960a253e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.296306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 818/1609] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.297652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.297949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.298259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.314213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c5c9e36a-b0f0-4ec6-91ce-c2960a253e9c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.317986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c5c9e36a-b0f0-4ec6-91ce-c2960a253e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.326706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 792/1610] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.330721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41adea-253b-473d-967c-ea2c78b54625 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.330721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41adea-253b-473d-967c-ea2c78b54625 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.330721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41adea-253b-473d-967c-ea2c78b54625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.335314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] MFA Rules not processed for user `79c1fa4f64dc42d9b2b3e3e7ac363343`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.338387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.355749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-bb41adea-253b-473d-967c-ea2c78b54625 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.357100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bb41adea-253b-473d-967c-ea2c78b54625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.366194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 793/1611] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.370283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.370573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.370885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.405367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] MFA Rules not processed for user `2e20671c2bde4268b88a66924290017a`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.406734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.425106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 819/1612] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.425106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.426497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.433470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:33.496778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 794/1613] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.497861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.505915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfcce7-59ec-4722-9da1-ff92f88ae677 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.507487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9fcc5-79b2-496b-9111-2a962d98fc20 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.511419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.518861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.519971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.520085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.520085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.520251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.520251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.520251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.525707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.527224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.533654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.536308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.539764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 795/1614] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:33.550661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.559024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 820/1615] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:33.559024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.559864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e839c-3869-498e-9a20-9a50fd0ed266 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.564360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.565015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.565303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.565588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.567846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4637f5-5972-4c83-9001-f98a4bb834ed service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.572244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.573654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.574007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.574288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.582064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.582064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.585455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.587337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831c227-96fb-4a4e-9668-8d54bb894c47 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.590166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 796/1616] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:33.591154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.598113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 821/1617] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:33.599386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b1bc-8f92-4cb6-9541-b5a35ab59633 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.599677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b1bc-8f92-4cb6-9541-b5a35ab59633 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.599959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b1bc-8f92-4cb6-9541-b5a35ab59633 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.601376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 822/1618] 10.222.0.74 () {54 vars in 961 bytes} [Wed May 25 14:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:33.603269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361a390-2828-4324-b428-395e60094a23 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.607662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.611462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.611462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.611692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.611692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.611692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.611692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.612058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.614008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.629410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:33.634341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.635625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.696742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 797/1619] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:33.697642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.705786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 823/1620] 10.222.0.74 () {58 vars in 1058 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:33.714530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.726396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b0266-43b5-4764-8e29-42b39451d6b7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.776932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664bb0e-c57d-442b-8b50-4ae298417134 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.776932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.776932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.777335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.778117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.779649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.783111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694508a-c661-459e-91d1-15fe5b3504fc admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.816451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.817505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.817883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.818381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.886035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 798/1621] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:33.889160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.889691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.890101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.903328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.905241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.918621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 824/1622] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:54:33.919725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.942769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8f135-4190-4b62-9708-60bbb42ff4e7 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.950344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:33.951149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:33.951686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:33.952098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:33.953280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:33.955665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:33.965815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] MFA Rules not processed for user `9eddf57ad0324f8dbde26150934df50a`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:33.971279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 825/1623] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:33.974720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:33.975846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:33.992059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b290b-0c03-4b84-ad43-57e882335e87 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:33.999961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.001765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.002966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.007416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] PATH_INFO: `/v3/projects/13b90a4126ad40f481e786f9cafd28dc/users/54abb351099747d992e0457074077386/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.026415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b90a4126ad40f481e786f9cafd28dc, user_id=54abb351099747d992e0457074077386, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.029530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.098565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 799/1624] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:34.103797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.121090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7053d6-8037-40a0-b556-9998bc79960a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.127758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 826/1625] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:33 2022] PUT /identity/v3/projects/13b90a4126ad40f481e786f9cafd28dc/users/54abb351099747d992e0457074077386/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:34.134971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.212015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.212649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.212893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.213175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.214723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.217310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.262673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8d3e-d870-43cf-871a-c023d1ff33fe admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.268444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:34.278681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.289628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.289628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.289628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.289628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.289628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.296394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 827/1626] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:34.301459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.325338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e62e6-9a08-40da-b4fd-4f5ae155b40e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.336602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.337744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.337982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.338160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] PATH_INFO: `/v3/projects/13b90a4126ad40f481e786f9cafd28dc/users/54abb351099747d992e0457074077386/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.340412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b90a4126ad40f481e786f9cafd28dc, user_id=54abb351099747d992e0457074077386, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.343283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e44a8-e51d-429f-8470-70e687e579e6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.351368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 800/1627] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:34 2022] POST /identity/v3/projects => generated 403 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:34.352824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.356873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 828/1628] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:34 2022] PUT /identity/v3/projects/13b90a4126ad40f481e786f9cafd28dc/users/54abb351099747d992e0457074077386/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:34.369033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097cbe-bab2-452b-a6a2-870b01faff5b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.370810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097cbe-bab2-452b-a6a2-870b01faff5b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.376177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097cbe-bab2-452b-a6a2-870b01faff5b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.376177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 829/1629] 10.222.0.74 () {54 vars in 912 bytes} [Wed May 25 14:54:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:54:34.376755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065b44d-b94e-473f-b591-029f05e37bfa None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.377999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065b44d-b94e-473f-b591-029f05e37bfa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.378232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065b44d-b94e-473f-b591-029f05e37bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.426516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3065b44d-b94e-473f-b591-029f05e37bfa None None] MFA Rules not processed for user `54abb351099747d992e0457074077386`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:34.428695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3065b44d-b94e-473f-b591-029f05e37bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:34.449146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2d89-bc05-4917-9dce-6f2ea70de44d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.531244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 830/1630] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:34.534858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.534858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.535137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.535252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.535809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.536098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.536381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.537095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.539698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.559447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:34.560784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:34.643273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 801/1631] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:34 2022] POST /identity/v3/users => generated 308 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:34.643273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.662944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 831/1632] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:34.663836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.677465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:34.678857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a8958-54ae-4728-bdee-80cfee5d2e80 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.701600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.702293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.702585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.702896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.703653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.705802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.738253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 832/1633] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:34.739903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.748872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a35f-a605-42ae-b9f2-500094c2f3fb admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.750895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22eb63-82e9-4e0f-95fa-a80dda975f0a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.753777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.754530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.754860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.755158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.756331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.757300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.757785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.758389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.760452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:34.775842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.777686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.790903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 833/1634] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:34.792033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:34.827341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af47fa-24fe-48af-9979-8d27fa6a42f0 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:34.846085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:34.851417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:34.851417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:34.851417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] PATH_INFO: `/v3/projects/9c4c5c36276a46c491c4044806059cb2/users/27eed8e13e6141db8698b474ef57a3cf/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:34.886288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4c5c36276a46c491c4044806059cb2, user_id=27eed8e13e6141db8698b474ef57a3cf, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.894014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.914348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:34.916038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:34.955215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 802/1635] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:35.026957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 834/1636] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:34 2022] PUT /identity/v3/projects/9c4c5c36276a46c491c4044806059cb2/users/27eed8e13e6141db8698b474ef57a3cf/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:35.038084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:35.068210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5ca34-c6c8-4dd8-a04a-b81a7865450c service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:35.203331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:35.204066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:35.204604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:35.204886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:35.205692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:35.208031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd8db-f08c-4d23-b899-51cba81bdb81 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:35.237167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 803/1637] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:35.240194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:35.260790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541d7a-cbcf-4172-9bf3-61f6cc62e176 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:35.276043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:35.276953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:35.277242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:35.277518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] PATH_INFO: `/v3/projects/9c4c5c36276a46c491c4044806059cb2/users/27eed8e13e6141db8698b474ef57a3cf/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:35.283183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4c5c36276a46c491c4044806059cb2, user_id=27eed8e13e6141db8698b474ef57a3cf, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:35.294849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6701f3-cbed-4eae-99fc-13809da0503f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:35.325528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 835/1638] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:35 2022] PUT /identity/v3/projects/9c4c5c36276a46c491c4044806059cb2/users/27eed8e13e6141db8698b474ef57a3cf/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:35.999055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:36.610059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaef3-0139-42a8-ab08-017827be300c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.610059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaef3-0139-42a8-ab08-017827be300c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.610059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaef3-0139-42a8-ab08-017827be300c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.647768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f123f-7af6-45fa-aca2-768554fb814e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.648905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f123f-7af6-45fa-aca2-768554fb814e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.649343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f123f-7af6-45fa-aca2-768554fb814e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.651072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 836/1639] 10.222.0.74 () {54 vars in 961 bytes} [Wed May 25 14:54:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:54:36.651693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1eddaef3-0139-42a8-ab08-017827be300c None None] MFA Rules not processed for user `8a96ba1833514a7ba5f17409d0781026`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:36.654444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1eddaef3-0139-42a8-ab08-017827be300c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:36.664149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b2956-6530-4975-a66f-e98815d0e8b3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.664966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b2956-6530-4975-a66f-e98815d0e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.665550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b2956-6530-4975-a66f-e98815d0e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.692241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8d7b2956-6530-4975-a66f-e98815d0e8b3 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:36.693824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8d7b2956-6530-4975-a66f-e98815d0e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:36.794278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 804/1640] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:36.801965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.804500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.804968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.851179 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 837/1641] 10.222.0.74 () {58 vars in 1058 bytes} [Wed May 25 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:36.853724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] MFA Rules not processed for user `8a96ba1833514a7ba5f17409d0781026`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:36.855385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:36.858893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992ac1d-e2b9-4f32-82ac-5ada74d902d9 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.859333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992ac1d-e2b9-4f32-82ac-5ada74d902d9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.859777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992ac1d-e2b9-4f32-82ac-5ada74d902d9 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.863195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 838/1642] 10.222.0.74 () {54 vars in 912 bytes} [Wed May 25 14:54:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:36.878363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b980-a8f7-4604-b305-9bfe89b26054 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.878363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b980-a8f7-4604-b305-9bfe89b26054 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.878792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b980-a8f7-4604-b305-9bfe89b26054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.886366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 805/1643] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:36.903835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:36.930728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47eafb-1b57-4e4b-ae2a-747824c08db9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:36.940167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:36.941084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:36.941723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:36.942429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:36.953947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:36.956730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:36.968463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 806/1644] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:54:36.987852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8551b980-a8f7-4604-b305-9bfe89b26054 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:36.989284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8551b980-a8f7-4604-b305-9bfe89b26054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:37.028448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 839/1645] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:37.029099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:37.136450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663bdba-02bf-4e98-a7c0-7674213a3259 service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:37.145212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:37.147270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:37.148619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:37.149240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:37.172681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:37.174816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c4d26-5f75-4f49-adc8-c73010777768 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:37.196088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 807/1646] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:37.732748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd55e-3427-4bd6-a852-4d6b30924ffc None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:37.733737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd55e-3427-4bd6-a852-4d6b30924ffc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:37.734862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd55e-3427-4bd6-a852-4d6b30924ffc None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:37.736731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 840/1647] 10.222.0.74 () {54 vars in 912 bytes} [Wed May 25 14:54:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:37.743465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229b503-bd97-44d1-8de0-a671fc6330b3 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:37.744863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229b503-bd97-44d1-8de0-a671fc6330b3 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:37.745316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229b503-bd97-44d1-8de0-a671fc6330b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:37.835850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:37.847376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4229b503-bd97-44d1-8de0-a671fc6330b3 None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:37.867025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4229b503-bd97-44d1-8de0-a671fc6330b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:37.984058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 808/1648] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:38.004712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decd55e-3427-4bd6-a852-4d6b30924ffc None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:38.019436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decd55e-3427-4bd6-a852-4d6b30924ffc None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:38.024419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:38.025248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:38.029117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:38.029117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:38.041959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:38.045346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dc04b-c779-447d-8138-b2f53960c0fc service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:38.052435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:38.063862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 841/1649] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:38.377937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fbb3b-c8ec-46f4-8fa4-c360b9b9daac None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:38.378314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fbb3b-c8ec-46f4-8fa4-c360b9b9daac None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:38.378737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fbb3b-c8ec-46f4-8fa4-c360b9b9daac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:38.440858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ce8fbb3b-c8ec-46f4-8fa4-c360b9b9daac None None] MFA Rules not processed for user `27eed8e13e6141db8698b474ef57a3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:38.442335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ce8fbb3b-c8ec-46f4-8fa4-c360b9b9daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:38.581237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 809/1650] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:38.746692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce3e8-7eb1-4117-90f6-a3539dc91fc0 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:38.749406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce3e8-7eb1-4117-90f6-a3539dc91fc0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:38.749406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce3e8-7eb1-4117-90f6-a3539dc91fc0 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:38.750202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 842/1651] 10.222.0.74 () {54 vars in 960 bytes} [Wed May 25 14:54:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:38.756286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:38.756769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:38.756910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:38.798631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:38.799927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:38.980809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 810/1652] 10.222.0.74 () {58 vars in 1057 bytes} [Wed May 25 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:39.000242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ce3e8-7eb1-4117-90f6-a3539dc91fc0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:39.014884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ce3e8-7eb1-4117-90f6-a3539dc91fc0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:39.021009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:39.031233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:39.031233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:39.031233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:39.031233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:39.047611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:39.050409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718de86-18e9-4b83-8450-2ff83dd945d4 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:39.062909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 843/1653] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:40.117413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:40.169054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:40.277028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf8b8b-5c75-4228-879f-e18338583d67 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:40.287906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:40.290975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:40.291502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:40.292073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] PATH_INFO: `/v3/users/2e20671c2bde4268b88a66924290017a` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:40.295739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e20671c2bde4268b88a66924290017a)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:40.299249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: WARNING py.warnings [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:40.299249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: warnings.warn(msg) May 25 14:54:40.299249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: May 25 14:54:40.304159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:40.510465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991710-7fb9-4bc0-ad72-dd455b5453e3 None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:40.511313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991710-7fb9-4bc0-ad72-dd455b5453e3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:40.511627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991710-7fb9-4bc0-ad72-dd455b5453e3 None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:40.513536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 844/1654] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:40.556450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e20671c2bde4268b88a66924290017a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:40.587887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e20671c2bde4268b88a66924290017a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:40.588672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:40.588778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e20671c2bde4268b88a66924290017a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:40.588967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:40.589254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:40.644782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:40.648066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:40.670128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e20671c2bde4268b88a66924290017a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:40.719997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 845/1655] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:54:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:40.757380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:40.782351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1793c-6fc5-40e2-8180-32c7506af6f1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:40.794975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-31b9edd7-b491-4a32-a857-bb26c5354ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e20671c2bde4268b88a66924290017a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:40.800215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:40.800893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:40.801178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:40.801740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:40.803732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:40.805816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 811/1656] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:54:40 2022] DELETE /identity/v3/users/2e20671c2bde4268b88a66924290017a => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:40.814533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4009-2846-4282-8339-29c47abc0ea9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:40.817803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4009-2846-4282-8339-29c47abc0ea9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:40.818150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4009-2846-4282-8339-29c47abc0ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:40.831279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:40.835500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:40.884181 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 846/1657] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:40.896028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0b4e4009-2846-4282-8339-29c47abc0ea9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:40.905138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0b4e4009-2846-4282-8339-29c47abc0ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:41.027837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 812/1658] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:41.105511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:41.107101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:41.141365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bebf1d-6d4f-4b9e-a79d-d7c57515e356 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:41.259657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:41.262847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:41.263976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:41.264590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:41.300946 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:41.302660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47cef9-4ff3-4345-aa21-73f99d0c15ec service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:41.333227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 847/1659] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:43.536335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:43.536938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:43.536938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:43.540212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 813/1660] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:43.550205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:43.551989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:43.552307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:43.604540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] MFA Rules not processed for user `dcd162deab164e70b629f8c88495ccb3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:43.605689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:43.721857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 848/1661] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:54:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:43.890882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:43.893209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:43.898630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:43.900093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:43.931269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bccdb-13f4-4929-b8d7-f0d5d67fa959 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:43.931365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d936f-9a84-4165-8c35-192db2c83b6a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:43.939701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:43.942616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:43.944939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:43.945943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] PATH_INFO: `/v3/users/9eddf57ad0324f8dbde26150934df50a` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:43.947896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eddf57ad0324f8dbde26150934df50a)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:43.949370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:43.952851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:43.953678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:43.954126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:43.954558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:43.992848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:44.006810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:44.026859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 814/1662] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:44.093752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:44.096007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:44.131998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5b210-5c6a-45fa-8377-ec19c34a3f05 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:44.140805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:44.140805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:44.140805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:44.140805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] PATH_INFO: `/v3/projects/23ff6c12f7604fe589925cd62611ccc3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:44.141054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ff6c12f7604fe589925cd62611ccc3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:44.142668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:44.201521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:44.369492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eddf57ad0324f8dbde26150934df50a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.381747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eddf57ad0324f8dbde26150934df50a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.382996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eddf57ad0324f8dbde26150934df50a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.385882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:44.399410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eddf57ad0324f8dbde26150934df50a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.448702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ff6c12f7604fe589925cd62611ccc3', 'request_id': 'req-696b67a0-7579-48e1-b3ff-b75ac956f750'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.513759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f32cf9d6-1174-43d9-8e27-406d22f985f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eddf57ad0324f8dbde26150934df50a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.525036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 849/1663] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:54:43 2022] DELETE /identity/v3/users/9eddf57ad0324f8dbde26150934df50a => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:44.527369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:44.527704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:44.528093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:44.580076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ff6c12f7604fe589925cd62611ccc3', 'request_id': 'req-696b67a0-7579-48e1-b3ff-b75ac956f750'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.589028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ff6c12f7604fe589925cd62611ccc3', 'request_id': 'req-696b67a0-7579-48e1-b3ff-b75ac956f750'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.620816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] The token cache is being invalidate because project 23ff6c12f7604fe589925cd62611ccc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:54:44.623716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.626033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-696b67a0-7579-48e1-b3ff-b75ac956f750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:44.631228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 815/1664] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:54:44 2022] DELETE /identity/v3/projects/23ff6c12f7604fe589925cd62611ccc3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:44.637059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa1ba-d32b-41ca-aedf-bfff2db0f1e9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:44.637547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa1ba-d32b-41ca-aedf-bfff2db0f1e9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:44.637975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faa1ba-d32b-41ca-aedf-bfff2db0f1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:44.655601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] MFA Rules not processed for user `cd25c0ad8e1c4fa08eca9b6bddeca55d`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:44.657761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:44.699223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b6faa1ba-d32b-41ca-aedf-bfff2db0f1e9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:44.702886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b6faa1ba-d32b-41ca-aedf-bfff2db0f1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:44.745521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 850/1665] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:44.760282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:44.762319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:44.877102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64cad8-6ea9-4fed-b826-10a9e855ae8d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:44.901494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:44.902146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:44.902439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:44.904337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:44.912456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:44.914084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:44.921622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 816/1666] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:44.923244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:44.923541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:44.923827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:44.935611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 851/1667] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:44.964927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:44.966201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:44.988213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 817/1668] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:44.992558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.013351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221e35-488d-4fc4-92d8-90fb7dd47f05 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.031880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.033102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.033383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.033732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.034454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:45.034960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.036747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.059062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.073582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 852/1669] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:45.074080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c0611-1092-4d68-8629-3d5d360c6f3d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.087576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.088319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.088626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.090078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.092963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.096357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.249270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 818/1670] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:45 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:45.254180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.270844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af829049-64a5-48b2-9f31-839482b56cd6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.339950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:45.407319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.408024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.411804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.412140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.412891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.414903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.622315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 853/1671] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:45 2022] POST /identity/v3/users => generated 290 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:45.625322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.636648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09a76-a555-462d-8918-2cddff87dc9c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.641438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.642851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.643061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.643344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.644083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.645759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.657060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 819/1672] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:45.660912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.675609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad53d0-fa75-4e36-9d28-b5ce1e9c31f1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.683414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.684137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.684424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.684707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.685447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.687188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.710009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 854/1673] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:45.710792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.731115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39db97d-d4ac-4ce2-bbaa-8305b8e912b7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.752169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.753315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.753728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.754336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] PATH_INFO: `/v3/projects/5178f98f4e554dd5af97fbe23ac522f5/users/954d166d36f14e7bb4897224d0cd94ef/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.786101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5178f98f4e554dd5af97fbe23ac522f5, user_id=954d166d36f14e7bb4897224d0cd94ef, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.791366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b03f24-f6a6-41a2-8e2f-d7e75219a7cd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.832679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:45.835706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:45.873504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dd634-12fe-4381-bb16-2034dd24fb5a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:45.890986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 820/1674] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:45 2022] PUT /identity/v3/projects/5178f98f4e554dd5af97fbe23ac522f5/users/954d166d36f14e7bb4897224d0cd94ef/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:45.906066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.914812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.914812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.917586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:45.920728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:45.921029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:45.921315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] PATH_INFO: `/v3/projects/66c7e071d94c432ba5b9b1ca57c0cafd` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:45.929706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c7e071d94c432ba5b9b1ca57c0cafd)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:45.935967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: WARNING py.warnings [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:54:45.935967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: warnings.warn(msg) May 25 14:54:45.935967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: May 25 14:54:45.942316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:45.999960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] MFA Rules not processed for user `954d166d36f14e7bb4897224d0cd94ef`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:46.004237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:46.080533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:46.161731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 821/1675] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:46.165069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:46.176680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c7e071d94c432ba5b9b1ca57c0cafd', 'request_id': 'req-b74c7c11-66e4-4007-a20c-f48098209bef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:46.284561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c7e071d94c432ba5b9b1ca57c0cafd', 'request_id': 'req-b74c7c11-66e4-4007-a20c-f48098209bef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:46.295241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c7e071d94c432ba5b9b1ca57c0cafd', 'request_id': 'req-b74c7c11-66e4-4007-a20c-f48098209bef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:46.335168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4445cd-7c76-4850-b5aa-cba2dcd666a4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:46.398500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] The token cache is being invalidate because project 66c7e071d94c432ba5b9b1ca57c0cafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:54:46.401065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:46.403297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:46.414623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 855/1676] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:54:45 2022] DELETE /identity/v3/projects/66c7e071d94c432ba5b9b1ca57c0cafd => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:46.416799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:46.420529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:46.551439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:46.552691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:46.553409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:46.553800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:46.556594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:46.575091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:46.668982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c7c11-66e4-4007-a20c-f48098209bef admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:46.682836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:46.687200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:46.687578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:46.687963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] PATH_INFO: `/v3/users/27eed8e13e6141db8698b474ef57a3cf` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:46.692016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27eed8e13e6141db8698b474ef57a3cf)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:46.695087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:46.705963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 822/1677] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:46 2022] POST /identity/v3/projects => generated 379 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:46.710302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:46.718833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:46.905381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ceef0-b12f-46bd-b756-86c9d11bf264 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:46.920078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:46.920826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:46.926240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:46.926598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:46.927446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:46.934237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:46.983897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27eed8e13e6141db8698b474ef57a3cf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:47.011690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27eed8e13e6141db8698b474ef57a3cf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:47.013450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27eed8e13e6141db8698b474ef57a3cf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:47.048956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27eed8e13e6141db8698b474ef57a3cf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:47.063194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 823/1678] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:46 2022] POST /identity/v3/users => generated 296 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:47.066590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:47.080438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc3d17-8df0-4ae3-8412-280d78c99fa7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:47.087048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:47.087963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:47.089742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:47.091652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:47.099481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:47.099481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:47.111483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 824/1679] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:47.112739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:47.123495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8504-c899-4cc6-b801-9eaad15f09e6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:47.129765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:47.130684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:47.130975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:47.131253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] PATH_INFO: `/v3/projects/7b2d510549cb4101a4b4c42e55508072/users/109c90503c2341fcbcfbdbfccdba0e98/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:47.159672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2d510549cb4101a4b4c42e55508072, user_id=109c90503c2341fcbcfbdbfccdba0e98, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:47.162950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:47.221432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27eed8e13e6141db8698b474ef57a3cf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:47.236966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 856/1680] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:54:46 2022] DELETE /identity/v3/users/27eed8e13e6141db8698b474ef57a3cf => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:47.324629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:47.325837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 825/1681] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:47 2022] PUT /identity/v3/projects/7b2d510549cb4101a4b4c42e55508072/users/109c90503c2341fcbcfbdbfccdba0e98/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:47.326577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:47.764931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:47.923502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1c797-8ab9-4cab-9197-fad73eb10e28 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:48.116286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:48.118315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:48.153458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:48.154974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:48.156026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.156026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.156187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.161635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:48.166628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:48.208556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 857/1682] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 884 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:48.210418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:48.255571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11d22-f8c7-4fb5-a8f5-28701c3d0626 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:48.274607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:48.275389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.279297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.287081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] PATH_INFO: `/v3/projects/7b2d510549cb4101a4b4c42e55508072/users/109c90503c2341fcbcfbdbfccdba0e98/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.287081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2d510549cb4101a4b4c42e55508072, user_id=109c90503c2341fcbcfbdbfccdba0e98, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:48.290624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a0ef4-caca-41bc-a65a-9a37476a8f05 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:48.297573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721185-8cb3-45ee-b85a-3a93133e58ca admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:48.304693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:48.305069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.305356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.305634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.309432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:48.317048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 858/1683] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:48 2022] PUT /identity/v3/projects/7b2d510549cb4101a4b4c42e55508072/users/109c90503c2341fcbcfbdbfccdba0e98/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:48.326996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.328485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.328593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.402906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] MFA Rules not processed for user `109c90503c2341fcbcfbdbfccdba0e98`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:48.404887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:48.567450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:48.572269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:48.631577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 826/1684] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:48.636591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 859/1685] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:48.645704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:48.648342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:48.695839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb731f03-7075-4d5e-90ed-a9ff33c3260e service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:48.743860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:48.745476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:48.884553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:48.889843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.893468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.893928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.948821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:48.950287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd778bc-74b1-4f49-a09b-44e450bed963 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:48.960044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcb963-50d1-419e-b1ab-33f533ddcb73 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:48.974801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:48.976336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:48.976455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:48.976748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] PATH_INFO: `/v3/projects/9c4c5c36276a46c491c4044806059cb2` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:48.980516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4c5c36276a46c491c4044806059cb2)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:49.007220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 827/1686] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:49.007754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:49.149944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:49.150486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:49.150486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:49.256521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] MFA Rules not processed for user `237510469a8e4e36ad40a34cdf705656`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:49.257795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.281568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4c5c36276a46c491c4044806059cb2', 'request_id': 'req-710c1b29-9ccb-47b5-879e-690b34eb6317'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:49.337728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:49.364423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4c5c36276a46c491c4044806059cb2', 'request_id': 'req-710c1b29-9ccb-47b5-879e-690b34eb6317'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:49.374798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4c5c36276a46c491c4044806059cb2', 'request_id': 'req-710c1b29-9ccb-47b5-879e-690b34eb6317'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:49.433798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] The token cache is being invalidate because project 9c4c5c36276a46c491c4044806059cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:54:49.434967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:49.437679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-710c1b29-9ccb-47b5-879e-690b34eb6317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:54:49.451012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 860/1687] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:54:48 2022] DELETE /identity/v3/projects/9c4c5c36276a46c491c4044806059cb2 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:54:49.452768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:49.454837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:49.455637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:49.460376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 828/1688] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:49.467220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:49.505491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:49.509403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.622233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477e11f-6baa-4e05-9fa3-1524f513acf1 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:49.627739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:49.628579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:49.628870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:49.629154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:49.631149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.702485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 861/1689] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:49.704656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:49.711886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.733062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:49.733542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:49.735451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec736c4b-49a9-4091-ae16-4dc63e1d658f service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:49.749003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 829/1690] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:49.760828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c122-f504-49a9-90cd-fe3bd9b86a67 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:49.761100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c122-f504-49a9-90cd-fe3bd9b86a67 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:49.761379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2c122-f504-49a9-90cd-fe3bd9b86a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:49.850108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-afc2c122-f504-49a9-90cd-fe3bd9b86a67 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:49.864095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-afc2c122-f504-49a9-90cd-fe3bd9b86a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.868225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164f16c-b0e2-4f4d-a2bb-0efe36245345 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:49.876097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:49.877099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:49.877531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:49.877963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:49.880094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:49.896262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:49.897919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:49.917021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 862/1692] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:49.917944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:49.927410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 830/1692] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:49.971442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478a79-8ca7-444b-b951-9a997a04111b service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.038530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.048086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.048805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.049098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.049887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:54:50.050209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:50.058682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:50.094041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.095195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.095948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.116128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 863/1693] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:54:50.118286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.151483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff7bf4-dc22-4895-b7f1-39ba0a0d756d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.174498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.177128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.177437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.177731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.179116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:50.180022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:50.181561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:50.182034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:50.342701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 864/1694] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:50 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:50.347831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.389609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 831/1695] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:54:50.533941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcae8-d2f8-4875-8b25-6835076d1996 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.551068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.551805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.552080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.552358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.553269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:50.558144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:50.717845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 865/1696] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:50 2022] POST /identity/v3/users => generated 290 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:50.722049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.772830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ad3d9-750d-4440-a104-99b052ced7f5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.788946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.789954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.790490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.791039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.791967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:50.795064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:50.864836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.879888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 832/1697] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:50.886229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913809-7a2f-48c1-a2d8-820e92629307 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.901438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.902267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.904548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.905052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.907699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:50.914833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:50.933965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.945042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 866/1698] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:50.949451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab9ad4-8a7d-4595-8b9a-abd4ef7778ba admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:50.956573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:50.957833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:50.958466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:50.958977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] PATH_INFO: `/v3/projects/4da69e1fcfdb4c7c98379aca4c500f64/users/49b3f800b7c44186855ab884d842bcdf/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:50.982903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:50.986832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:51.028704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da69e1fcfdb4c7c98379aca4c500f64, user_id=49b3f800b7c44186855ab884d842bcdf, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:51.036001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb35ea-2fc3-4359-88d9-541ca42feda4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:51.153369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 833/1699] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:50 2022] PUT /identity/v3/projects/4da69e1fcfdb4c7c98379aca4c500f64/users/49b3f800b7c44186855ab884d842bcdf/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:51.165393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:51.167985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:51.168303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:51.168585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:51.170589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6e5f5-58ed-49f2-bb95-64c11a8dc85b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:51.253075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] MFA Rules not processed for user `49b3f800b7c44186855ab884d842bcdf`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:51.255154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:51.341873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:51.342552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:51.343033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:51.343318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:51.404835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 834/1700] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:51.411763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:51.518507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:51.520348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:51.554091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 867/1701] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:54:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:54:51.593373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594b93d-1ad9-4c62-b3f8-d59650723f9f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:51.623256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:51.623993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:51.625981 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:51.626359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:51.628196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:51.633869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:51.866558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 835/1702] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:51 2022] POST /identity/v3/projects => generated 377 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:51.870787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:51.891221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e232-2620-4a81-888c-dc2ebdb63713 service swift] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:52.165045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:52.170878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:52.171665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.171955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.172279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.173563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:52.176690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:52.363558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 868/1703] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:51 2022] POST /identity/v3/users => generated 289 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:52.367751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:52.388005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef01e-a8e3-4d14-a491-c534fb9b34f9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:52.418069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:52.419384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.424699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.425071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.428490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:52.437662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:52.501919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 836/1704] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:52.504588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:52.520561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8175a0-9e9c-42d8-8e34-2c9001699118 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:52.540743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:52.546200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.547437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.547941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.548849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:52.550678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740eab9c-3c02-4a49-baa6-5ec1f3726154 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:52.583136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 869/1705] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:54:52.589735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:52.615522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d561776-a678-4b57-a894-aad6fbecdd1f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:52.646379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:52.647406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.647730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.652458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] PATH_INFO: `/v3/projects/20b84ba3c8514d119b1138fbdd1a06dd/users/ad28efc2d6dc49609de758f7b6743c84/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.730441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.731121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.731426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.744375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b84ba3c8514d119b1138fbdd1a06dd, user_id=ad28efc2d6dc49609de758f7b6743c84, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:52.755920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af2747-8775-41d6-9fdd-7f0b8f9d1e72 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:52.827791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:52.836132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 837/1706] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:52 2022] PUT /identity/v3/projects/20b84ba3c8514d119b1138fbdd1a06dd/users/ad28efc2d6dc49609de758f7b6743c84/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:54:52.850784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:52.851266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:52.851697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:52.871558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] MFA Rules not processed for user `9dba831ebb584de5a0c84f39901d3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:52.880324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:52.954447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] MFA Rules not processed for user `ad28efc2d6dc49609de758f7b6743c84`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:52.957714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:53.234698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 870/1707] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:53.246670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 838/1708] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:53.249267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:53.249405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:53.254199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:53.639425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5821a75-8abe-4287-85dd-15ad50c221af None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:53.653582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:53.654727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:53.655213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:53.655688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:53.656597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:53.659341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:53.678797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e93076-f5b2-4f1d-a8c0-3c43360630e7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:53.697882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:53.699626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:53.701111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:53.703767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:53.724334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:53.736600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:53.796295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 839/1709] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:54:53 2022] POST /identity/v3/projects => generated 379 bytes in 549 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:53.802753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:53.831879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 871/1710] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:53.993237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:54.005549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d9f76-a04b-4f0c-a48e-f6d600316c7a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:54.018777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:54.019675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.020069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.020457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.021370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:54.025076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:54.197951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 840/1711] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:54:53 2022] POST /identity/v3/users => generated 296 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:54:54.201266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:54.215718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2708-6261-4cc0-95c5-4d2c420971cd service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:54.221858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:54.224171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.224649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.225336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.226371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:54.228443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:54.244499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 872/1712] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:54:54.250868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:54.287000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb77c-c08f-4ef1-9853-aeb78f75c06d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:54.299643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:54.300908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.301248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.301584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] PATH_INFO: `/v3/projects/d71b2c770e2440a9a87c23f47af50540/users/95b3e83e02cb472b978a437b026010ca/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.365057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71b2c770e2440a9a87c23f47af50540, user_id=95b3e83e02cb472b978a437b026010ca, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:54.393503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:54.489387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:54.499739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 841/1713] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:54 2022] PUT /identity/v3/projects/d71b2c770e2440a9a87c23f47af50540/users/95b3e83e02cb472b978a437b026010ca/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:54:54.501866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:54.523372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef31b5-1278-4d13-9f59-a1fe5d6972f8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:54.660949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:54.665250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.665599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.665956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.670853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:54.674951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a74b1-7573-484b-97d3-61dfd7956fa3 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:54.739696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 873/1714] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:54:54.743713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:54.763884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a6a04-968a-44f9-8e5d-9dd68e1b34a3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:54.772451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:54.784884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.785556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.786370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] PATH_INFO: `/v3/projects/d71b2c770e2440a9a87c23f47af50540/users/95b3e83e02cb472b978a437b026010ca/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.789505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71b2c770e2440a9a87c23f47af50540, user_id=95b3e83e02cb472b978a437b026010ca, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:54.795487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:54.830539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 842/1715] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:54:54 2022] PUT /identity/v3/projects/d71b2c770e2440a9a87c23f47af50540/users/95b3e83e02cb472b978a437b026010ca/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:54:54.841470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920ef3-284d-483b-8dc2-01041a1daa1d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:54.846661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920ef3-284d-483b-8dc2-01041a1daa1d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:54.846661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920ef3-284d-483b-8dc2-01041a1daa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:54.916061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-9a920ef3-284d-483b-8dc2-01041a1daa1d None None] MFA Rules not processed for user `95b3e83e02cb472b978a437b026010ca`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:54.917459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9a920ef3-284d-483b-8dc2-01041a1daa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:55.155362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 874/1716] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:55.167197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:55.169570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:55.363367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634dfe4-b98c-4578-aeda-a89986febdb7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:55.374715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:55.375537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.375950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.376335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.389687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:55.392048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:55.410060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 843/1717] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:55.451839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:55.569872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb93a-8d51-42d7-a73b-e129d540ca73 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.574358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb93a-8d51-42d7-a73b-e129d540ca73 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.574358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb93a-8d51-42d7-a73b-e129d540ca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.590776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:55.602130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0d85-8bed-4972-9445-d40ca9ac82d9 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:55.613366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:55.614448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.615546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.616046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.649332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-661eb93a-8d51-42d7-a73b-e129d540ca73 None None] MFA Rules not processed for user `becb6feb52b643279066aef1a7a9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:55.654861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-661eb93a-8d51-42d7-a73b-e129d540ca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:55.809995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:55.811450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435680eb-33aa-407b-9f84-e40b487e97e9 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:55.833243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 844/1718] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:54:55.834792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ab95-c44e-4144-a841-9adfb48fe552 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.844878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ab95-c44e-4144-a841-9adfb48fe552 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.844878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4ab95-c44e-4144-a841-9adfb48fe552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.902301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 875/1719] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:55.912657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4ae4ab95-c44e-4144-a841-9adfb48fe552 None None] MFA Rules not processed for user `cd25c0ad8e1c4fa08eca9b6bddeca55d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:55.914140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4ae4ab95-c44e-4144-a841-9adfb48fe552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:55.915992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.916935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.917299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.986852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] MFA Rules not processed for user `becb6feb52b643279066aef1a7a9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:55.992972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:56.016051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 876/1720] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:56.027507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:56.037484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 845/1721] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:56.183521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46de839-b8b8-4ed5-9342-d89df3037959 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:56.206238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:56.207583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:56.208093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:56.208549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:56.231748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:56.236281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:56.269354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 877/1722] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:56.804714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:56.833997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9461f6-6af1-4fee-8879-cbdf13435827 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:56.835286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9461f6-6af1-4fee-8879-cbdf13435827 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:56.835645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9461f6-6af1-4fee-8879-cbdf13435827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:56.870819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ed9461f6-6af1-4fee-8879-cbdf13435827 None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:56.872606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ed9461f6-6af1-4fee-8879-cbdf13435827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:56.889899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:56.955033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 846/1723] 10.222.0.86 () {58 vars in 1015 bytes} [Wed May 25 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:57.048274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:57.051054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:57.072266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef9559-0f02-4e4d-a293-2de6f42ee436 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:57.086568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:57.089425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:57.089425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:57.089425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:57.097569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:57.101274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb5a5f-94f9-4316-bb08-2fc4a01232e3 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:57.150540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 878/1724] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:57.241389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:58.305528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:58.306546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:58.307114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:58.309168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 847/1725] 10.222.0.74 () {54 vars in 912 bytes} [Wed May 25 14:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:54:58.324246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fa94-7ffd-4534-8cff-bee900f66828 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:58.325702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fa94-7ffd-4534-8cff-bee900f66828 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:58.326425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fa94-7ffd-4534-8cff-bee900f66828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:58.371407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1dd9fa94-7ffd-4534-8cff-bee900f66828 None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:58.374870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1dd9fa94-7ffd-4534-8cff-bee900f66828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:58.406665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 879/1726] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:58.439843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:58.442017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:58.468440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b790ed-8dcf-4519-b48e-a5f1e28155c3 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:58.494900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:58.501791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:58.501791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:58.501791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:58.514987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:58.519641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ece6-100e-4827-87db-b0b2e73ff518 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:58.542761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 848/1727] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:54:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:59.784691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd208-0976-4e48-852e-6e0a0df5568d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:59.785270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd208-0976-4e48-852e-6e0a0df5568d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:59.785344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd208-0976-4e48-852e-6e0a0df5568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:59.835252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-895cd208-0976-4e48-852e-6e0a0df5568d None None] MFA Rules not processed for user `becb6feb52b643279066aef1a7a9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:59.837091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-895cd208-0976-4e48-852e-6e0a0df5568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:59.867420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 880/1728] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:54:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:00.450581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbfdb-f9b6-4f6b-9677-158a83b0157f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:00.452843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbfdb-f9b6-4f6b-9677-158a83b0157f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:00.456145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecbfdb-f9b6-4f6b-9677-158a83b0157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:00.570155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-71ecbfdb-f9b6-4f6b-9677-158a83b0157f None None] MFA Rules not processed for user `237510469a8e4e36ad40a34cdf705656`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:00.592532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-71ecbfdb-f9b6-4f6b-9677-158a83b0157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:00.847444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 849/1729] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:01.676935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08864d3-9ad8-4044-81fd-3704ba92f58f None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:01.679920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08864d3-9ad8-4044-81fd-3704ba92f58f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:01.680395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08864d3-9ad8-4044-81fd-3704ba92f58f None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:01.682474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 881/1730] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:55:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:55:01.822721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442b81-0dfc-4ef0-ad1f-6e39f3ae4b3d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:01.823277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442b81-0dfc-4ef0-ad1f-6e39f3ae4b3d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:01.823454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442b81-0dfc-4ef0-ad1f-6e39f3ae4b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:01.856385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-41442b81-0dfc-4ef0-ad1f-6e39f3ae4b3d None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:01.858522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-41442b81-0dfc-4ef0-ad1f-6e39f3ae4b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:01.925101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 850/1731] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:55:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:02.071040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08864d3-9ad8-4044-81fd-3704ba92f58f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:02.097138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08864d3-9ad8-4044-81fd-3704ba92f58f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:02.122109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:02.123127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:02.123663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:02.124118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:02.134710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:02.136500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:02.149978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 882/1732] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:03.136081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765714f6-8351-464a-8a34-f4779cb861dd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:03.139592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765714f6-8351-464a-8a34-f4779cb861dd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:03.140802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765714f6-8351-464a-8a34-f4779cb861dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:03.181070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-765714f6-8351-464a-8a34-f4779cb861dd None None] MFA Rules not processed for user `9dba831ebb584de5a0c84f39901d3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:03.182592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-765714f6-8351-464a-8a34-f4779cb861dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:03.285764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 851/1733] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:12.148530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=23 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:13.276657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=76 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:14.440886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:14.601688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0dd1-ddb7-4c85-8059-f43fdec453a6 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:14.618281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:14.620274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:14.628597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:14.628597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] PATH_INFO: `/v3/users/49b3f800b7c44186855ab884d842bcdf` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:14.628597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b3f800b7c44186855ab884d842bcdf)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:14.628597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:14.925682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b3f800b7c44186855ab884d842bcdf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:14.939558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b3f800b7c44186855ab884d842bcdf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:14.940469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b3f800b7c44186855ab884d842bcdf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:14.971350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b3f800b7c44186855ab884d842bcdf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:15.153121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b3f800b7c44186855ab884d842bcdf'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:15.165431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 883/1734] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:14 2022] DELETE /identity/v3/users/49b3f800b7c44186855ab884d842bcdf => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:15.168183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:15.168681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:15.168789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:15.194197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:15.195690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:15.315418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 852/1735] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:15.326773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:15.329848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:15.435363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce47643-0f1e-4f83-ae19-3ed765d7cb3f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:15.435363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:15.435363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:15.435363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:15.435854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:15.449847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:15.452158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:15.466456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 884/1736] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:16.269413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:16.271148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:16.285379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e822989-bce6-465b-ac92-2763d1adf668 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:16.293224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:16.294456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:16.294979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:16.295460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] PATH_INFO: `/v3/projects/4da69e1fcfdb4c7c98379aca4c500f64` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:16.296992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da69e1fcfdb4c7c98379aca4c500f64)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:16.299690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:16.508255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da69e1fcfdb4c7c98379aca4c500f64', 'request_id': 'req-92eba11a-07f9-49e2-b4d4-57567a676ec7'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:16.613094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:16.615177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:16.626869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da69e1fcfdb4c7c98379aca4c500f64', 'request_id': 'req-92eba11a-07f9-49e2-b4d4-57567a676ec7'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:16.632656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41aca6-2bce-4c46-8a05-cc1118a7ac4d service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:16.635518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da69e1fcfdb4c7c98379aca4c500f64', 'request_id': 'req-92eba11a-07f9-49e2-b4d4-57567a676ec7'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:16.656900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] The token cache is being invalidate because project 4da69e1fcfdb4c7c98379aca4c500f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:16.657193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:16.662739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:16.664093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:16.667342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 853/1737] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:16 2022] DELETE /identity/v3/projects/4da69e1fcfdb4c7c98379aca4c500f64 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:16.671579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:16.673167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:16.773923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:16.774854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:16.775145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:16.775424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] PATH_INFO: `/v3/users/954d166d36f14e7bb4897224d0cd94ef` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:16.783588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] RBAC: Authorizing `identity:delete_user(user_id=954d166d36f14e7bb4897224d0cd94ef)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:16.786358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:16.827119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eba11a-07f9-49e2-b4d4-57567a676ec7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:16.833546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:16.834579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:16.836176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:16.836176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] PATH_INFO: `/v3/users/ad28efc2d6dc49609de758f7b6743c84` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:16.836573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad28efc2d6dc49609de758f7b6743c84)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:16.838670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:16.997468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:17.120763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954d166d36f14e7bb4897224d0cd94ef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.135567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad28efc2d6dc49609de758f7b6743c84'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.135889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954d166d36f14e7bb4897224d0cd94ef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.138359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954d166d36f14e7bb4897224d0cd94ef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.154785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad28efc2d6dc49609de758f7b6743c84'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.157420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad28efc2d6dc49609de758f7b6743c84'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.172121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954d166d36f14e7bb4897224d0cd94ef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.194648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad28efc2d6dc49609de758f7b6743c84'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.295135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c9d1becc-70ea-4b5a-acd9-2a5d08a6895b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954d166d36f14e7bb4897224d0cd94ef'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.314680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 885/1738] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:16 2022] DELETE /identity/v3/users/954d166d36f14e7bb4897224d0cd94ef => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:17.314680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:17.314680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:17.314680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:17.357258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:17.357258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:17.380313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad28efc2d6dc49609de758f7b6743c84'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:17.390761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 854/1739] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:16 2022] DELETE /identity/v3/users/ad28efc2d6dc49609de758f7b6743c84 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:17.426231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 886/1740] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:17.436041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:17.437424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:17.514198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8122d1-22d8-4f9e-8814-60932eae3465 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:17.518227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:17.518943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:17.519226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:17.519504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:17.578366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:17.579775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:17.589213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 855/1741] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:17.857738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:17.860206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:17.889913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff81a1-3a38-4d3e-914d-3c145bb17cbb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:17.906699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:17.907447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:17.907901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:17.908241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] PATH_INFO: `/v3/projects/20b84ba3c8514d119b1138fbdd1a06dd` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:17.910041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b84ba3c8514d119b1138fbdd1a06dd)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:17.913280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:18.071739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b84ba3c8514d119b1138fbdd1a06dd', 'request_id': 'req-a16b458e-2476-4006-99db-1db00fe9ba32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.085805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:18.087247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:18.111893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8289a3-5cf1-4526-b318-e8d72e3d119f service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:18.146128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b84ba3c8514d119b1138fbdd1a06dd', 'request_id': 'req-a16b458e-2476-4006-99db-1db00fe9ba32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.152714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b84ba3c8514d119b1138fbdd1a06dd', 'request_id': 'req-a16b458e-2476-4006-99db-1db00fe9ba32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.165176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] The token cache is being invalidate because project 20b84ba3c8514d119b1138fbdd1a06dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:18.165725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.166717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.169043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 887/1742] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:17 2022] DELETE /identity/v3/projects/20b84ba3c8514d119b1138fbdd1a06dd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:18.172973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:18.175102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:18.177303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:18.178346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:18.178657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:18.179013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] PATH_INFO: `/v3/projects/5178f98f4e554dd5af97fbe23ac522f5` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:18.180272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5178f98f4e554dd5af97fbe23ac522f5)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:18.182279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:18.188949 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b458e-2476-4006-99db-1db00fe9ba32 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:18.195892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:18.195892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:18.198993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:18.199489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] PATH_INFO: `/v3/users/95b3e83e02cb472b978a437b026010ca` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:18.201033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b3e83e02cb472b978a437b026010ca)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:18.203583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:18.306416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5178f98f4e554dd5af97fbe23ac522f5', 'request_id': 'req-e41f5236-14f6-4a23-8b7c-9068c1afaa89'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.458815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5178f98f4e554dd5af97fbe23ac522f5', 'request_id': 'req-e41f5236-14f6-4a23-8b7c-9068c1afaa89'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.466282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5178f98f4e554dd5af97fbe23ac522f5', 'request_id': 'req-e41f5236-14f6-4a23-8b7c-9068c1afaa89'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.477409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] The token cache is being invalidate because project 5178f98f4e554dd5af97fbe23ac522f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:18.477697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.479057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.480154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 856/1743] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:18 2022] DELETE /identity/v3/projects/5178f98f4e554dd5af97fbe23ac522f5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:18.485305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:18.486964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:18.504162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b3e83e02cb472b978a437b026010ca'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.512442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b3e83e02cb472b978a437b026010ca'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.513316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b3e83e02cb472b978a437b026010ca'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.532952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b3e83e02cb472b978a437b026010ca'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.557274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f5236-14f6-4a23-8b7c-9068c1afaa89 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:18.561867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:18.562770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:18.563053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:18.563331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] PATH_INFO: `/v3/users/109c90503c2341fcbcfbdbfccdba0e98` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:18.564534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=109c90503c2341fcbcfbdbfccdba0e98)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:18.567949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:18.625938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b3e83e02cb472b978a437b026010ca'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.633093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 888/1744] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:18 2022] DELETE /identity/v3/users/95b3e83e02cb472b978a437b026010ca => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:18.810908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109c90503c2341fcbcfbdbfccdba0e98'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.817869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109c90503c2341fcbcfbdbfccdba0e98'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.818561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109c90503c2341fcbcfbdbfccdba0e98'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.831319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109c90503c2341fcbcfbdbfccdba0e98'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.927403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109c90503c2341fcbcfbdbfccdba0e98'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:18.939848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 857/1745] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:18 2022] DELETE /identity/v3/users/109c90503c2341fcbcfbdbfccdba0e98 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:19.089303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:19.226427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.228266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.314557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0936a8-7771-46b6-8dab-f1e57dbe5bc9 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.370998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.372652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.375273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.375273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.375273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.375273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] PATH_INFO: `/v3/projects/d71b2c770e2440a9a87c23f47af50540` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.376155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71b2c770e2440a9a87c23f47af50540)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.378697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.390762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076b3e6-2c24-485e-a2ef-007dc4a3f41b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.398389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:19.414944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.414944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.414944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.414944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] PATH_INFO: `/v3/projects/7b2d510549cb4101a4b4c42e55508072` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.415414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2d510549cb4101a4b4c42e55508072)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.416361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.512692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71b2c770e2440a9a87c23f47af50540', 'request_id': 'req-f346a448-e194-4c19-9a03-6b24c2bb0e66'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.538515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2d510549cb4101a4b4c42e55508072', 'request_id': 'req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.578516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71b2c770e2440a9a87c23f47af50540', 'request_id': 'req-f346a448-e194-4c19-9a03-6b24c2bb0e66'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.585856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71b2c770e2440a9a87c23f47af50540', 'request_id': 'req-f346a448-e194-4c19-9a03-6b24c2bb0e66'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.600395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] The token cache is being invalidate because project d71b2c770e2440a9a87c23f47af50540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:19.600843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.601564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f346a448-e194-4c19-9a03-6b24c2bb0e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.608557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 889/1746] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:19 2022] DELETE /identity/v3/projects/d71b2c770e2440a9a87c23f47af50540 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:19.610152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b966-157e-42e8-bc04-395bb8baad55 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.610544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b966-157e-42e8-bc04-395bb8baad55 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.610951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b966-157e-42e8-bc04-395bb8baad55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.632326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a408b966-157e-42e8-bc04-395bb8baad55 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:19.633910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a408b966-157e-42e8-bc04-395bb8baad55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.652830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2d510549cb4101a4b4c42e55508072', 'request_id': 'req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.654604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2d510549cb4101a4b4c42e55508072', 'request_id': 'req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.666563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] The token cache is being invalidate because project 7b2d510549cb4101a4b4c42e55508072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:19.666563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.667285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee790d6e-59bd-49b2-b4b6-8c4d0b95d4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:19.672221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 858/1747] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:19 2022] DELETE /identity/v3/projects/7b2d510549cb4101a4b4c42e55508072 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:19.676478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055f8d-7837-4c4d-bdf3-fa7b633d4dfc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.676800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055f8d-7837-4c4d-bdf3-fa7b633d4dfc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.677163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055f8d-7837-4c4d-bdf3-fa7b633d4dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.700808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4e055f8d-7837-4c4d-bdf3-fa7b633d4dfc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:19.702396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e055f8d-7837-4c4d-bdf3-fa7b633d4dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.710040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 890/1748] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:19.711231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 859/1749] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:19.718429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.718744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.719026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.720279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.721449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.721449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.740890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:19.741511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:19.742116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.743279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:19.750678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 860/1750] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:19.755500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 891/1751] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:19.756348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.757011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.766391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8531-89ee-4e65-a018-a4cbae444221 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.767140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e03010-28a1-4220-a87d-68cffa3e8ad1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.771652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.772414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.772586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.773017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.773322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.773492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.773582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.773926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.774421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:19.774846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.777694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:19.777694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.777898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.777981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.790606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 861/1752] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:19.790837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 892/1752] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:55:19.795457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.798920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.808891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c104fd-9529-45eb-a5e1-8a32c45103ea admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.809879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38750db3-6195-4976-8d3b-0205646ed26f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:19.814040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.814587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:19.814974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.815351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:19.815446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.815637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:19.815919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.816341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:19.816885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.817364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:19.819013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.819655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:19.940899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 893/1753] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:19.944978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:19.989150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 862/1754] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:19.990191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.007497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d902fff-2ba7-4776-8f60-f17e59875601 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.052547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38128-6628-4d42-804b-987a5499b82c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.057232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.057902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.058188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.058466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.059196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.061114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.062688 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.063365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.063653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.063938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.064635 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.066866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.162510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 863/1755] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:20.166310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.177061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42756a3-45e5-4420-8a22-91307eac2d12 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.181826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.182536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.182906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.183367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.184126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.185722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.199476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 864/1756] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:20.200898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.208165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 894/1757] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:19 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:20.215152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bff93a-7aed-48a3-a1d2-8913cbd64da5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.217191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.217868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.218156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.218438 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.218855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.219242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.220980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.230228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ba9d-8ed8-4e03-9594-b27ae6b6b9a2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.233509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 865/1758] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:20.237852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.239777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.240789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.243708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.243708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.243708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.244517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.248255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c052-ff9c-4a54-94b1-c625f90a1522 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.255558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 895/1759] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:20.257112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.258008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.258390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.258700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] PATH_INFO: `/v3/projects/36ce0d58f4074846b6af9c21337430a7/users/efddb2b5a93740149f689d39e6445355/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.259525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.270426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d12ec-1ea0-41b3-a48d-61a74f8052f5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.278238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.278374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ce0d58f4074846b6af9c21337430a7, user_id=efddb2b5a93740149f689d39e6445355, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.278476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.278476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.278476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.278476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.279287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014cd6f-f357-4f9b-9f8d-59b6c3740316 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.280166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.292433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 896/1760] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:20.302833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.312446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279abc7e-fa94-4a95-834a-9458e92f725a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.317316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.318187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.318468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.318783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] PATH_INFO: `/v3/projects/8dea89b258094e00b9687e5d855e0a22/users/bc055ea1914f4386ab3e4ce9264a5fe1/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.333756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dea89b258094e00b9687e5d855e0a22, user_id=bc055ea1914f4386ab3e4ce9264a5fe1, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.336645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209493d-c722-41ad-9695-4b3327ccd613 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.420663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 866/1761] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:20 2022] PUT /identity/v3/projects/36ce0d58f4074846b6af9c21337430a7/users/efddb2b5a93740149f689d39e6445355/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:20.432545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.433106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.433397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.461745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] MFA Rules not processed for user `efddb2b5a93740149f689d39e6445355`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:20.463179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:20.492252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 897/1762] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:20 2022] PUT /identity/v3/projects/8dea89b258094e00b9687e5d855e0a22/users/bc055ea1914f4386ab3e4ce9264a5fe1/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:20.495312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.495612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.495905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.526937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] MFA Rules not processed for user `bc055ea1914f4386ab3e4ce9264a5fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:20.528258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:20.540693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 867/1763] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:20.541858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.613287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 898/1765] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:20.617672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.620864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728118c3-6804-49bc-ba88-35b043f8a013 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.666383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:20.689287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.689287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.689287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.689287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.689917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.692133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.714624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41f35-799c-4296-b5d8-2624940b2068 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.720823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.721528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.721814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.722213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.722996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.724936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.738847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:20.779722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 868/1766] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/projects => generated 367 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:20.785977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.845836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c2985-ee5e-4ed1-8330-fc9f025d1ef8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.850264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 899/1767] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/projects => generated 363 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:20.850461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.851229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.851526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.851794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.852508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.854186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.855006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.915236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6068-ced4-4731-bedf-8f0f2b7b189f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.919975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.920640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.920919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.921192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.921871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.923798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.940566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 869/1768] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:20.944889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.954805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f1af-0ff4-45da-9af0-e5b85eb90d7d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.963395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.964097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.964386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:20.964668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:20.965403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:20.967042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:20.981574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 870/1769] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:20.982712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:20.993424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d104a9c-4a36-48e0-b111-bbd7879a67d8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:20.998328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:20.999359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:20.999714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.000087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] PATH_INFO: `/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444/users/7f696070cad2476b9e128c9c5eb134d0/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.015932 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e6bd9b8cd54eaabbee04c3b7828444, user_id=7f696070cad2476b9e128c9c5eb134d0, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.019298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.034789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 900/1770] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:20 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:21.036139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.046166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fc14b-a6a0-4a9d-a789-ea93a32a083a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.051851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.052779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.053238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.053967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.054839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.056672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.070677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 901/1771] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:21.072127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.084082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2e93f-031f-4ff6-b8d9-588f1c8d3181 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.088808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.090132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.090132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.090425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] PATH_INFO: `/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961/users/06a0ad605db1436285228ef8fd463c57/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.105354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc66444ffc346f0beabc4f3a6a5e961, user_id=06a0ad605db1436285228ef8fd463c57, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.108079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.133774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 871/1772] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:20 2022] PUT /identity/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444/users/7f696070cad2476b9e128c9c5eb134d0/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:21.137635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.199165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3c52a-ec38-4f4f-9752-6aa1a8861781 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.203750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.204414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.204698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.204975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.205699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.207362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.220194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 872/1773] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:21.221200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.221320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 902/1774] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:21 2022] PUT /identity/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961/users/06a0ad605db1436285228ef8fd463c57/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:21.225064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.231085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf0d0d-7da0-45b9-8306-b74e7c40b222 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.286673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7ce5b-c860-442b-8800-02070ed516b9 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.289749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.290879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.291289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.292016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] PATH_INFO: `/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444/users/7f696070cad2476b9e128c9c5eb134d0/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.293725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e6bd9b8cd54eaabbee04c3b7828444, user_id=7f696070cad2476b9e128c9c5eb134d0, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.293844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.294495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.294981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.295093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.295799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.296599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.297391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ff77b-a4a2-459d-b1e3-3a1e7a594fee admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.308121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 873/1775] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:21 2022] PUT /identity/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444/users/7f696070cad2476b9e128c9c5eb134d0/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:21.309622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 903/1776] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:21.313930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.327492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.327963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.328316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.358005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] MFA Rules not processed for user `7f696070cad2476b9e128c9c5eb134d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:21.359371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:21.400477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cebc2b-3c3b-42ca-bbf9-bd7583078cdc admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.406930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.407826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.408100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.408374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] PATH_INFO: `/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961/users/06a0ad605db1436285228ef8fd463c57/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.411455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc66444ffc346f0beabc4f3a6a5e961, user_id=06a0ad605db1436285228ef8fd463c57, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.414114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0b007-1f20-4492-bedc-8839d41486c9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.430725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 874/1777] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:21 2022] PUT /identity/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961/users/06a0ad605db1436285228ef8fd463c57/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:21.438710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.438992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.439433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.445629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 904/1778] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:21.456224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.457648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:21.467799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] MFA Rules not processed for user `06a0ad605db1436285228ef8fd463c57`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:21.469156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:21.517890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c3c8-c3b4-40bc-a9c2-732e967c17d3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.544092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 875/1779] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:21.556246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.557609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:21.567922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.568780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.570139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:21.570595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.571091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.615191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3801c-dd9d-4558-87bc-f25779eea058 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.618860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.619718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.620358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.622378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:21.622378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.631538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.632523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.633022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.634028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:21.642726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 876/1780] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:21.646168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 905/1781] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:21.904201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:21.913094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b005cb9-3a73-47c8-a74d-c67a1d600f77 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:21.916757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:21.917398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:21.917681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:21.917961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:21.989904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:21.991452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff1f1c-7a66-4e3c-916c-0d39fdd90aa7 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:22.001174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 877/1782] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:22.055334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:22.118205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:22.756310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:22.872027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:23.479853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:23.593704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f1bb4-a06c-48ec-b531-8d712a9b25c6 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:23.600650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:23.601712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:23.602002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:23.602281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] PATH_INFO: `/v3/users/bc055ea1914f4386ab3e4ce9264a5fe1` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:23.608173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc055ea1914f4386ab3e4ce9264a5fe1)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:23.616740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:23.858114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc055ea1914f4386ab3e4ce9264a5fe1'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:23.870547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc055ea1914f4386ab3e4ce9264a5fe1'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:23.871620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc055ea1914f4386ab3e4ce9264a5fe1'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:23.888764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc055ea1914f4386ab3e4ce9264a5fe1'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:23.961398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc055ea1914f4386ab3e4ce9264a5fe1'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:23.978150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 906/1783] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:23 2022] DELETE /identity/v3/users/bc055ea1914f4386ab3e4ce9264a5fe1 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:23.978953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89a107-f2ca-4861-b7d8-962fbbe450cf None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:23.979402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89a107-f2ca-4861-b7d8-962fbbe450cf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:23.979749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89a107-f2ca-4861-b7d8-962fbbe450cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:23.991964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:24.002580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ab89a107-f2ca-4861-b7d8-962fbbe450cf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:24.004106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ab89a107-f2ca-4861-b7d8-962fbbe450cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:24.114300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 878/1784] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:24.168311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:24.169881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:24.252071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:24.399710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59b71-d8bb-4e77-a8c7-f3f44be8f726 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:24.406201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:24.407210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:24.409061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:24.410626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:24.437332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:24.441367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc3e24-c2a6-44f4-ad5d-95e91e79c16c service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:24.455519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 907/1785] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:24.600789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:24.600789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:24.600789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:24.600789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 879/1786] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:55:24.617497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:24.618925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:24.619376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:24.707057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:24.717291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:24.914590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 908/1787] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:55:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:25.084832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:25.427351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:25.429134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:25.466945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f6f14-75f8-4bfc-9eef-d23accec43d7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:25.476047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:25.482831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:25.482831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:25.482831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:25.491359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:25.493626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:25.517060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 880/1788] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:25.967059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:25.968492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:25.995215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82de1-03eb-41f5-ac64-22f8062e40c7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:26.005672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:26.005672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:26.005672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:26.005672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] PATH_INFO: `/v3/projects/8dea89b258094e00b9687e5d855e0a22` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:26.006093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dea89b258094e00b9687e5d855e0a22)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:26.006093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:26.157364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dea89b258094e00b9687e5d855e0a22', 'request_id': 'req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.266047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dea89b258094e00b9687e5d855e0a22', 'request_id': 'req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.274151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dea89b258094e00b9687e5d855e0a22', 'request_id': 'req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.289709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] The token cache is being invalidate because project 8dea89b258094e00b9687e5d855e0a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:26.290001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.291010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.296262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 909/1789] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:25 2022] DELETE /identity/v3/projects/8dea89b258094e00b9687e5d855e0a22 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:26.297231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:26.310312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27d353-8f61-4a95-ad88-ce698a2d82ac service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:26.372571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:26.373492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:26.373781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:26.374062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] PATH_INFO: `/v3/users/06a0ad605db1436285228ef8fd463c57` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:26.375277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a0ad605db1436285228ef8fd463c57)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:26.377520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:26.596556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:26.607590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a0ad605db1436285228ef8fd463c57'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.615650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a0ad605db1436285228ef8fd463c57'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.622448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a0ad605db1436285228ef8fd463c57'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.638794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a0ad605db1436285228ef8fd463c57'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.732762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bccc7d84-f966-4fab-a1bb-1ba55f67dbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a0ad605db1436285228ef8fd463c57'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:26.739444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 881/1790] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:26 2022] DELETE /identity/v3/users/06a0ad605db1436285228ef8fd463c57 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:27.291628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:27.293139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:27.306031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdeb4-84c1-49a6-a700-6f5d852c4a27 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:27.363586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:27.364469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:27.364776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:27.365035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] PATH_INFO: `/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:27.366225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc66444ffc346f0beabc4f3a6a5e961)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:27.368236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:27.484374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:27.503508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc66444ffc346f0beabc4f3a6a5e961', 'request_id': 'req-14412e5a-9e2d-489c-a31b-aebee34c03c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:27.612430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc66444ffc346f0beabc4f3a6a5e961', 'request_id': 'req-14412e5a-9e2d-489c-a31b-aebee34c03c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:27.619527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc66444ffc346f0beabc4f3a6a5e961', 'request_id': 'req-14412e5a-9e2d-489c-a31b-aebee34c03c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:27.633736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] The token cache is being invalidate because project 4bc66444ffc346f0beabc4f3a6a5e961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:27.634032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:27.634810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-14412e5a-9e2d-489c-a31b-aebee34c03c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:27.636350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 910/1791] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:27 2022] DELETE /identity/v3/projects/4bc66444ffc346f0beabc4f3a6a5e961 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:27.641232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:27.641665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:27.641813 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:27.664214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:27.665468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:27.743805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 882/1792] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:27.751899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:27.752285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:27.753213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:27.777603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:27.779503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:27.807129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 911/1793] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:27.808979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:27.821157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed7a52-bfad-4e9f-8e1b-0b9165be768b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:27.832137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:27.832814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:27.833264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:27.833536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:27.834738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:27.835057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:27.836692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:27.849656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 883/1794] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:27.855197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:27.866252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57511d1a-8a7b-4416-a04d-55c64ab428c9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:27.872092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:27.873005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:27.873267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:27.873644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:27.874462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:27.876613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:27.990760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 912/1795] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:27 2022] POST /identity/v3/projects => generated 357 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:27.992093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.004019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1912ff-f3c5-490f-8938-b5d4a5b6e6fd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.096553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.097055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.102191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.102191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.102191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.102191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.228497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 884/1796] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:27 2022] POST /identity/v3/users => generated 279 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:28.229706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.240851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23629894-bd23-4c4c-ae99-c7817a688a43 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.245401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.246066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.246353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.246654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.247405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.249161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.259893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 913/1797] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:28.263935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.275861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ab75-22cf-4941-ac86-328b1ebfbc0f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.280877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.281512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.281947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.282230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.282997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.285191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978b77d-7ec6-4f67-bd7b-291ff1e4161b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.296602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 885/1798] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:28.305475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.316381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7653a0-23fb-488e-ae8e-9a0d5ce62fbe admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.320983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.321899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.322308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.322589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] PATH_INFO: `/v3/projects/01ba559017f1442ea8a636e3c749a4b7/users/8a666d23da3342b099202779665a3e50/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.338140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ba559017f1442ea8a636e3c749a4b7, user_id=8a666d23da3342b099202779665a3e50, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.345369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.506121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 914/1799] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:28 2022] PUT /identity/v3/projects/01ba559017f1442ea8a636e3c749a4b7/users/8a666d23da3342b099202779665a3e50/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:28.516544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.516958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.517331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.545034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] MFA Rules not processed for user `8a666d23da3342b099202779665a3e50`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:28.546262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:28.555882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:28.614428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 886/1800] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:28.619128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.679132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174b3e0-2929-4223-827b-34a8793dd9cd admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.683835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.684497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.684784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.685064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.685757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.687729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.808200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 915/1801] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:28 2022] POST /identity/v3/projects => generated 357 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:28.812408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.823340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998273-e166-41d0-980d-d1497edf5bde None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.877600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.878277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.878564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.878886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.879580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.881500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:28.967765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 887/1802] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:28 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:28.971980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:28.982776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9347b-2d09-4538-b205-ceb811701338 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:28.987463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:28.988335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:28.988770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:28.989193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:28.990054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:28.992209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.004410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 916/1803] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:29.008684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:29.019720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c5df-8a7e-451f-adb7-b17f91e18c21 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:29.025359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:29.026250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.026528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.026840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] PATH_INFO: `/v3/projects/0869acb011fe4aa1a46394751d41ae5f/users/e1b9b35da028475ca032e657634f3400/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.043190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0869acb011fe4aa1a46394751d41ae5f, user_id=e1b9b35da028475ca032e657634f3400, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:29.046140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.204934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 888/1804] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:29 2022] PUT /identity/v3/projects/0869acb011fe4aa1a46394751d41ae5f/users/e1b9b35da028475ca032e657634f3400/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:29.209029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:29.219953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dee61-1c2d-4e8d-9dcb-1bfc3c69a1ad admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:29.232779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:29.281497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:29.282434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.282926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.283352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.284267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:29.287567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e08128-0e4f-434c-b662-394bc9f95a1e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.298454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 917/1805] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:29.303271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:29.313826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6bf26-c331-45f0-b3fe-4c6ecfe241bd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:29.319617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:29.320434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.320885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.321311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] PATH_INFO: `/v3/projects/0869acb011fe4aa1a46394751d41ae5f/users/e1b9b35da028475ca032e657634f3400/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.323527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0869acb011fe4aa1a46394751d41ae5f, user_id=e1b9b35da028475ca032e657634f3400, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:29.326359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.346502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 889/1806] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:29 2022] PUT /identity/v3/projects/0869acb011fe4aa1a46394751d41ae5f/users/e1b9b35da028475ca032e657634f3400/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:29.359338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.359837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.359928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.394894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] MFA Rules not processed for user `e1b9b35da028475ca032e657634f3400`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:29.396262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:29.509074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 918/1807] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:29.521243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:29.523153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:29.612320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d9fde-8c40-472a-a786-be1735f3157b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:29.617066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:29.617820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.618357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.619382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.632403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:29.634970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.647609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 890/1808] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:29.911240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:29.921882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2fc9e-e07a-48fa-a674-aa35d8ed99c5 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:29.926280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:29.926962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:29.927250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:29.932634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:29.983235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:29.984701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350822b-b223-4e70-8008-cec897cc2700 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:29.994197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 919/1809] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:32.005540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:32.007040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:32.013607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:32.022603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316216-b3ae-494b-8d42-ced655caee4b None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.023145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316216-b3ae-494b-8d42-ced655caee4b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.023596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316216-b3ae-494b-8d42-ced655caee4b None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.025472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 920/1810] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:55:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:55:32.036603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.037524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.037820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.057823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:32.059067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:32.086928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd4cb-69f3-4a68-b02e-14b9ff7d4d18 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:32.091916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:32.092787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.093077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.093348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] PATH_INFO: `/v3/users/efddb2b5a93740149f689d39e6445355` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.094533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efddb2b5a93740149f689d39e6445355)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:32.097120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:32.098134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:32.154081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 921/1811] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:32.317579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:32.319079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:32.333866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f5eba-785b-436a-895b-da50590a45fc None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:32.340869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:32.341694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.342201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.342389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.353043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:32.354663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44be395-9135-49ba-80fd-6911e7f01894 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:32.365490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 922/1812] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:32.406044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efddb2b5a93740149f689d39e6445355'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:32.413728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efddb2b5a93740149f689d39e6445355'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:32.414420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efddb2b5a93740149f689d39e6445355'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:32.426991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efddb2b5a93740149f689d39e6445355'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:32.515088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efddb2b5a93740149f689d39e6445355'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:32.522128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 891/1813] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:32 2022] DELETE /identity/v3/users/efddb2b5a93740149f689d39e6445355 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:32.530793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.531019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.531300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.549247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:32.550451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:32.612124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 923/1814] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:32.621748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:32.623175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:32.680687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a377d7-51f2-429b-ab1e-11395d43bfa5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:32.684352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:32.684996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:32.685281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:32.685559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:32.695670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:32.697159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:32.707346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 892/1815] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:33.417363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:33.418895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:33.431344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7d61f-e24d-4fc8-a76a-39b7ac9469f4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:33.436141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:33.436987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:33.437272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:33.437552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] PATH_INFO: `/v3/projects/36ce0d58f4074846b6af9c21337430a7` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:33.438820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ce0d58f4074846b6af9c21337430a7)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:33.440793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:33.655388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ce0d58f4074846b6af9c21337430a7', 'request_id': 'req-a090d250-cede-4ab1-abdf-3cd2c4f175e5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:33.847136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ce0d58f4074846b6af9c21337430a7', 'request_id': 'req-a090d250-cede-4ab1-abdf-3cd2c4f175e5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:33.853419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ce0d58f4074846b6af9c21337430a7', 'request_id': 'req-a090d250-cede-4ab1-abdf-3cd2c4f175e5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:33.865750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] The token cache is being invalidate because project 36ce0d58f4074846b6af9c21337430a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:33.866127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:33.866834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:33.868419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 924/1816] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:33 2022] DELETE /identity/v3/projects/36ce0d58f4074846b6af9c21337430a7 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:33.872747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:33.882293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b14e4d-e072-4a22-8526-01af07a0189b service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:33.913285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:33.945421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:33.946309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:33.955965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:33.955965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] PATH_INFO: `/v3/users/7f696070cad2476b9e128c9c5eb134d0` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:33.955965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f696070cad2476b9e128c9c5eb134d0)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:33.955965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:34.166706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f696070cad2476b9e128c9c5eb134d0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:34.174830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f696070cad2476b9e128c9c5eb134d0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:34.174830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f696070cad2476b9e128c9c5eb134d0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:34.187606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f696070cad2476b9e128c9c5eb134d0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:34.349846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5e3a5bb4-d01a-4c01-9c48-739b42b5f85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f696070cad2476b9e128c9c5eb134d0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:34.357122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 893/1817] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:33 2022] DELETE /identity/v3/users/7f696070cad2476b9e128c9c5eb134d0 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:34.824359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:34.825831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:34.841805 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090d250-cede-4ab1-abdf-3cd2c4f175e5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:34.884993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:34.903260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:34.904130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:34.904425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:34.904757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] PATH_INFO: `/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:34.905936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e6bd9b8cd54eaabbee04c3b7828444)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:34.908032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.008545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e6bd9b8cd54eaabbee04c3b7828444', 'request_id': 'req-71dda277-e034-424a-a53a-79af0a5ea9b2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.109021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e6bd9b8cd54eaabbee04c3b7828444', 'request_id': 'req-71dda277-e034-424a-a53a-79af0a5ea9b2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.115595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e6bd9b8cd54eaabbee04c3b7828444', 'request_id': 'req-71dda277-e034-424a-a53a-79af0a5ea9b2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.126944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] The token cache is being invalidate because project 20e6bd9b8cd54eaabbee04c3b7828444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:35.127271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.127882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-71dda277-e034-424a-a53a-79af0a5ea9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.129459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 925/1818] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:34 2022] DELETE /identity/v3/projects/20e6bd9b8cd54eaabbee04c3b7828444 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:35.134370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.134851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.135036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.153882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:35.155140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:35.225476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 894/1819] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:35.235984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.236301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.237532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.268056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:35.268056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:35.272588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 926/1820] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:35.277158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.289053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed8fe0-e47c-4580-8c81-3e23dab0f585 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.294052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.294761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.295067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.303041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.303041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:35.303041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.303041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.312432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 895/1821] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:35.317413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.327652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a6f1-bd86-4dc5-9f6d-328df7a611fa None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.333594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.334753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.334753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.334890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.335540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.337570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.552378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 927/1822] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:35 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:35.556500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.568149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896c2eb-8796-46c3-8698-1d8f913aee85 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.599706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:35.627667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.628344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.628627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.628904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.629663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.631692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.712675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.715564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:35.729627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e395c-1147-4781-898f-99a9ff95b935 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.736158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.737074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.737356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.737632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] PATH_INFO: `/v3/users/79c1fa4f64dc42d9b2b3e3e7ac363343` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.738859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c1fa4f64dc42d9b2b3e3e7ac363343)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.740791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.762542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 896/1823] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:35 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:35.766142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.776676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f59ebe-17cb-4866-ba44-127c637e5f7d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.781891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.782565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.782877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.783255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.786191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.786191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.796488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 897/1824] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:35.800265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.821001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd6d00-8bf9-45ce-a755-7fc4b839d89b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.846937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.847759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.848116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.848468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.849277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.851058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.872206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 898/1825] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:35.878798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:35.894992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab108d-424e-426f-ba64-f88d81694fa3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:35.899922 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:35.901092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:35.901383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:35.901665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] PATH_INFO: `/v3/projects/bf731c0ba1cd4c5a8b91fcf592b5d98b/users/8f41ba9a7def47c7a7a7af48216168f7/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:35.920087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf731c0ba1cd4c5a8b91fcf592b5d98b, user_id=8f41ba9a7def47c7a7a7af48216168f7, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:35.923063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c57f96-6351-4546-b969-e399e4e96d16 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:35.972214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c1fa4f64dc42d9b2b3e3e7ac363343'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.979423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c1fa4f64dc42d9b2b3e3e7ac363343'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.980223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c1fa4f64dc42d9b2b3e3e7ac363343'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:35.993410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c1fa4f64dc42d9b2b3e3e7ac363343'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:36.074836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 899/1826] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:35 2022] PUT /identity/v3/projects/bf731c0ba1cd4c5a8b91fcf592b5d98b/users/8f41ba9a7def47c7a7a7af48216168f7/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:36.083775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.084073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.084354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.104112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b6bca37f-e07d-4b06-afef-49e4501aa8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c1fa4f64dc42d9b2b3e3e7ac363343'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:36.114950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 928/1827] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:35 2022] DELETE /identity/v3/users/79c1fa4f64dc42d9b2b3e3e7ac363343 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:36.119990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.120290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.120582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.121038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] MFA Rules not processed for user `8f41ba9a7def47c7a7a7af48216168f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:36.122294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:36.138564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:36.139860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:36.148753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:36.192953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 900/1828] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:36.197391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.198684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:36.203387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 929/1829] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:36.213303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7deb1-2462-4431-bf50-134264621a39 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.213514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.214760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:36.218730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.223869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.223869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.223869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.223869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.224415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.271713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3dd2-41b0-4c3f-a696-75517cf87de1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.277059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.277705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.277986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.278265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.288749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.290206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.300786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 930/1830] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:36.356404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 901/1831] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:36 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:36.374618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.388741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69913d-2953-4f2e-9a9f-313215bdc034 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.469723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.470401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.470722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.471029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.471724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.473752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.579101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 931/1832] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:36 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:36.581446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.597002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09e189-5ae8-497a-815a-600878c765a5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.609718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.617819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.617819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.617819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.617819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.617819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.636500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 902/1833] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:36.640719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.675481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20a2f-960f-4667-bfa9-8c51a44b35b0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.675481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.675481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.675481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.677282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] PATH_INFO: `/v3/projects/ef70337bc6f44cf99240f44b6008c962/users/90beba61984c49d3a7a164a1d2225e41/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.703436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef70337bc6f44cf99240f44b6008c962, user_id=90beba61984c49d3a7a164a1d2225e41, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.708169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.841689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 932/1834] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:36 2022] PUT /identity/v3/projects/ef70337bc6f44cf99240f44b6008c962/users/90beba61984c49d3a7a164a1d2225e41/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:36.855703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.867097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5983bc-4d09-4a8c-b775-d13a65ffd076 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.900004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:36.926463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.927263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.927607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.927894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.928622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.930210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.941028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 903/1835] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:36.947593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.948679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:36.952181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:36.962042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b4002-e72e-4a37-b9c9-32149c640205 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.965328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff60f2-43cb-45e7-a76a-a7bb8b64fad0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:36.967088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.968153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.968786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.968786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] PATH_INFO: `/v3/projects/ef70337bc6f44cf99240f44b6008c962/users/90beba61984c49d3a7a164a1d2225e41/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.970873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef70337bc6f44cf99240f44b6008c962, user_id=90beba61984c49d3a7a164a1d2225e41, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.973657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8dcb9-60ef-41a5-9bb7-82c7e3438282 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.974331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:36.975211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:36.975586 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:36.976212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] PATH_INFO: `/v3/projects/79333a7f533640e99bd21b9c8be032f7` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:36.978063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=79333a7f533640e99bd21b9c8be032f7)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:36.980742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:36.984316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 933/1836] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:36 2022] PUT /identity/v3/projects/ef70337bc6f44cf99240f44b6008c962/users/90beba61984c49d3a7a164a1d2225e41/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:36.995458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:37.002960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:37.002960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:37.024964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] MFA Rules not processed for user `90beba61984c49d3a7a164a1d2225e41`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:37.026384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:37.100246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 934/1837] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:37.113542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79333a7f533640e99bd21b9c8be032f7', 'request_id': 'req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:37.116247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:37.119312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:37.184816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79333a7f533640e99bd21b9c8be032f7', 'request_id': 'req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:37.193556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79333a7f533640e99bd21b9c8be032f7', 'request_id': 'req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:37.199750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdef9f1-ab11-4636-87d9-64a42aafa024 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:37.207383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] The token cache is being invalidate because project 79333a7f533640e99bd21b9c8be032f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:37.208051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:37.209259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:37.213339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 904/1838] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:36 2022] DELETE /identity/v3/projects/79333a7f533640e99bd21b9c8be032f7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:37.218136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:37.220257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:37.255102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:37.255772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:37.256061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:37.256377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:37.258309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:37.291699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50d72-d2cb-4495-a0bf-092ecb9974fe admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:37.300338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:37.301365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:37.301750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:37.302132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] PATH_INFO: `/v3/users/54abb351099747d992e0457074077386` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:37.304137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54abb351099747d992e0457074077386)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:37.307439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:37.328319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:37.329804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:37.343054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 935/1839] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:38.098788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:38.284254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54abb351099747d992e0457074077386'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:38.301721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54abb351099747d992e0457074077386'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:38.306963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54abb351099747d992e0457074077386'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:38.347018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54abb351099747d992e0457074077386'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:38.383342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:38.510820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54abb351099747d992e0457074077386'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:38.527150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 905/1840] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:37 2022] DELETE /identity/v3/users/54abb351099747d992e0457074077386 => generated 0 bytes in 1309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:38.875836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:38.879734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:38.905395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981bb1-d2a2-44c6-956c-01dca5f01a2e service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:38.973448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:38.973448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:38.973448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:38.973448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] PATH_INFO: `/v3/users/8f41ba9a7def47c7a7a7af48216168f7` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:38.974652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f41ba9a7def47c7a7a7af48216168f7)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:38.976786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:39.188110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f41ba9a7def47c7a7a7af48216168f7'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.195790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f41ba9a7def47c7a7a7af48216168f7'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.200916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f41ba9a7def47c7a7a7af48216168f7'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.207930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f41ba9a7def47c7a7a7af48216168f7'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.232108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:39.233598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.247466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edeeb2f-f3c4-455b-a1e1-1e7cda19fbbd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:39.253401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:39.254310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.254520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.254825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] PATH_INFO: `/v3/projects/13b90a4126ad40f481e786f9cafd28dc` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.256086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b90a4126ad40f481e786f9cafd28dc)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:39.258102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:39.294494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-51db2c9b-2096-43fc-b090-eb1e4e3c14d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f41ba9a7def47c7a7a7af48216168f7'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.302040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 936/1841] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:38 2022] DELETE /identity/v3/users/8f41ba9a7def47c7a7a7af48216168f7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:39.306028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.306324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.306611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.327117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:39.328374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.388659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:39.397808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 937/1842] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:39.407561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:39.409452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.410490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b90a4126ad40f481e786f9cafd28dc', 'request_id': 'req-6164854f-b125-42ad-8932-b7556f931205'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.465725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8c793-f6bf-478a-b762-6bd81150410f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:39.492075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b90a4126ad40f481e786f9cafd28dc', 'request_id': 'req-6164854f-b125-42ad-8932-b7556f931205'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.500525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b90a4126ad40f481e786f9cafd28dc', 'request_id': 'req-6164854f-b125-42ad-8932-b7556f931205'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.514206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] The token cache is being invalidate because project 13b90a4126ad40f481e786f9cafd28dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:39.514487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.515204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6164854f-b125-42ad-8932-b7556f931205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:39.516771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 906/1843] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:39 2022] DELETE /identity/v3/projects/13b90a4126ad40f481e786f9cafd28dc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:39.525405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:39.525405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.525605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.525711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.525793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.525861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.525933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.527455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.544054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:39.551072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.593742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:39.595264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4724b3-7edb-49e7-829f-78ed2b369103 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:39.605655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 938/1844] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:39.612974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 907/1845] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:39.621156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.621568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.621953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.639632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:39.640849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:39.649971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 939/1846] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:39.654133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:39.664790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc52853-ebdc-4fe5-8b66-c19f0eb10b2c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:39.669754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:39.670500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.670973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.671413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.672320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:39.672871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:39.675314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:39.699510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 908/1847] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:39.702782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:39.714111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5a94f-10df-4f16-b13a-d6eedbabf5df None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:39.719615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:39.720290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.720573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.721388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.722092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:39.724261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:39.897658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 940/1848] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:39 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:39.914319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:39.918435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd294a0-3a7d-4f12-9994-33ed71460c89 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:39.992119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:39.992951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:39.993614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:39.993909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:39.994671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:39.996633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.101537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 909/1849] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:39 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:40.108779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.119281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842b84d-84ab-495e-bede-128df06838cb admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.123940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.124606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.124894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.125178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.125945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.131417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.145857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 941/1850] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:40.150467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.163530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a442a-07e4-4ed2-bcb8-9a36ba08e8ed admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.168314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.168980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.169260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.169540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.170363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.172163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.183698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 910/1851] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:40.187680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.200184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acb8d9-0402-4167-a506-f2f86ae70b6b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.208514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.209272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.210231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.210432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:40.210704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.210995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] PATH_INFO: `/v3/projects/df02dcad7ee04acaa239746abbd879af/users/09e4625160b34514b287a878885ab9da/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.232565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e9c9-08c1-4755-a603-eebbd1f08817 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.238256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df02dcad7ee04acaa239746abbd879af, user_id=09e4625160b34514b287a878885ab9da, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.238516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.238897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49658a-f709-4587-9110-604e8ba06c48 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.239135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.239422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.239751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] PATH_INFO: `/v3/projects/bf731c0ba1cd4c5a8b91fcf592b5d98b` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.240973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf731c0ba1cd4c5a8b91fcf592b5d98b)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.243725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.362080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:40.368845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 942/1852] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:40 2022] PUT /identity/v3/projects/df02dcad7ee04acaa239746abbd879af/users/09e4625160b34514b287a878885ab9da/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:40.380594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.380921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.381202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.426887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] MFA Rules not processed for user `09e4625160b34514b287a878885ab9da`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:40.428123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:40.505692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf731c0ba1cd4c5a8b91fcf592b5d98b', 'request_id': 'req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:40.530911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 943/1853] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:40.532077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.585693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf731c0ba1cd4c5a8b91fcf592b5d98b', 'request_id': 'req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:40.602098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf731c0ba1cd4c5a8b91fcf592b5d98b', 'request_id': 'req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:40.625717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] The token cache is being invalidate because project bf731c0ba1cd4c5a8b91fcf592b5d98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:40.625986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:40.628309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:40.631047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 911/1854] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:40 2022] DELETE /identity/v3/projects/bf731c0ba1cd4c5a8b91fcf592b5d98b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:40.635986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.637863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:40.669344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb01bd-3342-458c-ad45-3804497cd9ac None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.788316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.789210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.790928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.793281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.794156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.798978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.806097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17c61-07a4-4275-a5fe-9d70d624ee2f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:40.811994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:40.816197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:40.816197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:40.816197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] PATH_INFO: `/v3/users/90beba61984c49d3a7a164a1d2225e41` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:40.816197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=90beba61984c49d3a7a164a1d2225e41)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:40.816945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:40.925621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 944/1855] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:40 2022] POST /identity/v3/projects => generated 369 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:40.927335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:40.930063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:41.042514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:41.081671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698a8e2-c9a2-452c-a073-4a17008e5e2c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:41.095523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:41.098386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.100618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.100618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.102290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:41.105677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90beba61984c49d3a7a164a1d2225e41'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:41.106331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:41.124666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90beba61984c49d3a7a164a1d2225e41'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:41.126282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90beba61984c49d3a7a164a1d2225e41'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:41.140755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90beba61984c49d3a7a164a1d2225e41'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:41.242619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 945/1856] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:40 2022] POST /identity/v3/users => generated 291 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:41.246410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.262255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ac060-9145-42de-a82c-9f0b7349eb27 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:41.268369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:41.269091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.269295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.269525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.271860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:41.273266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90beba61984c49d3a7a164a1d2225e41'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:41.283926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 912/1857] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:40 2022] DELETE /identity/v3/users/90beba61984c49d3a7a164a1d2225e41 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:41.292799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:41.318319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 946/1858] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:41.318871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.321030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:41.412023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b42123-a495-41c3-9a6b-5824e3bbeecc admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:41.418284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:41.419516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.420490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.420967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] PATH_INFO: `/v3/projects/88b833508eac4a4caaef4a57b12c333c/users/da2b97e8457240f1a8c689c8bdcdddd3/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.442515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b833508eac4a4caaef4a57b12c333c, user_id=da2b97e8457240f1a8c689c8bdcdddd3, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:41.448548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:41.585886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 913/1859] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:41 2022] PUT /identity/v3/projects/88b833508eac4a4caaef4a57b12c333c/users/da2b97e8457240f1a8c689c8bdcdddd3/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:41.590044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.600891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131fb00-0823-4367-be59-4e2974a951d0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:41.674816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:41.676328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.676440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.676705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.677437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:41.680723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9204af3-91b7-492d-bde8-9f8b8206b2b7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:41.702813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 947/1860] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:41.706864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.718613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1dc34-04c5-4f78-ba3b-634c6ac7638f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:41.723848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:41.724618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.724740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.725183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] PATH_INFO: `/v3/projects/88b833508eac4a4caaef4a57b12c333c/users/da2b97e8457240f1a8c689c8bdcdddd3/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.727077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b833508eac4a4caaef4a57b12c333c, user_id=da2b97e8457240f1a8c689c8bdcdddd3, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:41.729861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:41.741137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 914/1861] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:41 2022] PUT /identity/v3/projects/88b833508eac4a4caaef4a57b12c333c/users/da2b97e8457240f1a8c689c8bdcdddd3/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:41.757365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:41.757666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:41.757957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:41.790780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] MFA Rules not processed for user `da2b97e8457240f1a8c689c8bdcdddd3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:41.792861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:41.901961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 948/1862] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:41.913331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.915527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:41.967998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:41.969865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:42.001351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb592e-15c5-4369-88b7-e3d45ee823ce admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.006303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.007179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.007777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.008236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.025365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.027762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.057621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 915/1863] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:42.060145 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b7c7f-264a-4e21-813a-f205ca29d074 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.073692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.074659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.074986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.075268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] PATH_INFO: `/v3/projects/ef70337bc6f44cf99240f44b6008c962` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.076526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef70337bc6f44cf99240f44b6008c962)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.078428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.196683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef70337bc6f44cf99240f44b6008c962', 'request_id': 'req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:42.199254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.200714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:42.213163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178a093-083f-48d7-8a93-3aa13586c614 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.263405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.264078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.264361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.264636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.266417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:42.269051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef70337bc6f44cf99240f44b6008c962', 'request_id': 'req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:42.275363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef70337bc6f44cf99240f44b6008c962', 'request_id': 'req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:42.286438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] The token cache is being invalidate because project ef70337bc6f44cf99240f44b6008c962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:42.286850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:42.287578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f7dce3d8-8b1e-4131-a9f1-09a93f0b51cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:42.289252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 949/1864] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:41 2022] DELETE /identity/v3/projects/ef70337bc6f44cf99240f44b6008c962 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:42.294077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.294368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.294730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.313707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:42.315629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:42.329270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.330883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c640c2-55cd-425e-b783-9713e736b4b6 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.340426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 916/1865] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:42.375943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 950/1866] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:42.384435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.384729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.385015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.402387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:42.403637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:42.412384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 917/1867] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:42.416709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.426892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d3ab2-6f21-46eb-b4ac-266e08c5a87c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.431767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.432426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.432715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.432995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.433732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:42.434045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.435803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.448084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 951/1868] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:42.452648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.461999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c552d7-c1a1-4fe2-be41-040c7d3b2af2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.468906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.469924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.470102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.470394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.471676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.474344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.590521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 918/1869] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:42 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:42.594657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.604668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f977d-b22f-401d-aa38-808d59049ab8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.663524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.664851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.664851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.664851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.665447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.667412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.773368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 952/1870] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:42 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:42.778762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.789593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf483c3-b06c-4546-bbd9-1a09049d9125 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.794521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.795227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.795925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.795925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.796519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.798531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.811029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 919/1871] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:42.815948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.826664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba331d-4cc5-46d2-82ff-cda61bd8d692 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.832544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.833356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.833757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.834112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.835006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.836753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc427f5-f05e-4447-9214-1f8c46206da7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.852276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:42.852785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 953/1872] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:42.864499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fbe69-f039-4919-8898-fbe7ba95462d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:42.877958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:42.879071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:42.879361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:42.879642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] PATH_INFO: `/v3/projects/1469f74b311a471f88066b489011944b/users/af3d27a232e5498fa7545fd7b2aef56c/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:42.896722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1469f74b311a471f88066b489011944b, user_id=af3d27a232e5498fa7545fd7b2aef56c, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:42.899638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:42.985738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:43.021365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 920/1873] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:42 2022] PUT /identity/v3/projects/1469f74b311a471f88066b489011944b/users/af3d27a232e5498fa7545fd7b2aef56c/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:43.031892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.032677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.032855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.085760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] MFA Rules not processed for user `af3d27a232e5498fa7545fd7b2aef56c`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:43.088106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:43.160052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 954/1874] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:43.164622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.237831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c13b0-d3df-4446-955e-211784d208eb admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.242955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.243623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.244247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.244247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.245013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.246968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.381790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 921/1875] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:43 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:43.385646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.397168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753941-4971-4fe6-ab0e-44d45d59c09c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.453689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.454365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.454682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.454992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.455684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.457706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.494594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:43.545867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 955/1876] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:43 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:43.550408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.560823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de12fc-902b-4fcf-bd30-5af77b620ad6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.567293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.567950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.568234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.568603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.569541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.571207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.584220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 922/1877] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:43.588543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.599468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469660f0-fe8c-4024-a11c-e1a9597c5ade admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.605772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.605772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.605772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.605772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] PATH_INFO: `/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1/users/094cab1f24584e54965c79a6b3111216/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.621579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4bf9fe4d2a4c19aacb61f8ef59fda1, user_id=094cab1f24584e54965c79a6b3111216, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.624387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.683180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 956/1878] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:43 2022] PUT /identity/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1/users/094cab1f24584e54965c79a6b3111216/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:43.691568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.701476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60d830-b7a3-4537-9717-3d73eaa95a73 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.758583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.759215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.759506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.759787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.760506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.762186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f29e8f-0b52-44d6-8162-992a8877eced admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.773392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 923/1879] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:43.777299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.788188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57babce-1bd4-45f5-ab96-58f757bc5610 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:43.793093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:43.794053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.794405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.794800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] PATH_INFO: `/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1/users/094cab1f24584e54965c79a6b3111216/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.797093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4bf9fe4d2a4c19aacb61f8ef59fda1, user_id=094cab1f24584e54965c79a6b3111216, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:43.799909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:43.812661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 957/1880] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:43 2022] PUT /identity/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1/users/094cab1f24584e54965c79a6b3111216/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:43.825217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:43.825511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:43.825794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:43.883412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] MFA Rules not processed for user `094cab1f24584e54965c79a6b3111216`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:43.884658 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:43.956014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 924/1881] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:43.967888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:43.969875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:44.046448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac010d-469c-43f0-9a39-79fc8c7471ce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:44.051197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:44.051522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:44.051879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:44.052218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:44.061769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:44.063255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:44.078820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 958/1882] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:44.232846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:44.259213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:44.260635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:44.282083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583d51d-7905-4898-aa7c-241e39ad9451 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:44.282083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:44.282083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:44.282083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:44.282532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:44.376347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:44.378552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6be6ee-6990-41ca-83a0-3d8f60ade1d4 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:44.388190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 925/1883] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:44.831623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:44.972078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:45.038807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1dbf-5319-4233-a55e-edd16b55d3d0 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:45.045813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:45.047031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:45.047213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:45.047703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] PATH_INFO: `/v3/users/af3d27a232e5498fa7545fd7b2aef56c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:45.049498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3d27a232e5498fa7545fd7b2aef56c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:45.051573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:45.087054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:45.263382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3d27a232e5498fa7545fd7b2aef56c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:45.271484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3d27a232e5498fa7545fd7b2aef56c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:45.272322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3d27a232e5498fa7545fd7b2aef56c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:45.303452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3d27a232e5498fa7545fd7b2aef56c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:45.412138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3d27a232e5498fa7545fd7b2aef56c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:45.419334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 959/1884] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:44 2022] DELETE /identity/v3/users/af3d27a232e5498fa7545fd7b2aef56c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:45.424040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:45.424336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:45.424617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:45.443122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:45.444560 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:45.511044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 926/1885] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:45.529236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:45.531082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:45.591752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caa477-6baa-4953-9536-52723446e86c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:45.595427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:45.596111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:45.596394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:45.596728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:45.606599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:45.608249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:45.618473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 960/1886] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:45.634758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:45.797094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:46.167944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:46.169935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:46.183179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ec52b-1d0e-443a-901b-5318f4182fbd None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:46.187892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:46.188833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:46.189126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:46.189430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] PATH_INFO: `/v3/projects/1469f74b311a471f88066b489011944b` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:46.191124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1469f74b311a471f88066b489011944b)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:46.193059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:46.288618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:46.370330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1469f74b311a471f88066b489011944b', 'request_id': 'req-5f77b65b-84be-4d50-85c9-630d4a084626'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.458079 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1469f74b311a471f88066b489011944b', 'request_id': 'req-5f77b65b-84be-4d50-85c9-630d4a084626'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.464824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1469f74b311a471f88066b489011944b', 'request_id': 'req-5f77b65b-84be-4d50-85c9-630d4a084626'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.477070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:46.482669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] The token cache is being invalidate because project 1469f74b311a471f88066b489011944b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:46.482669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.484159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.485753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 927/1887] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:46 2022] DELETE /identity/v3/projects/1469f74b311a471f88066b489011944b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:46.490362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:46.500933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310fa5e-e930-45e5-992d-49474e78e006 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:46.578386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:46.579370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:46.579655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:46.580472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] PATH_INFO: `/v3/users/094cab1f24584e54965c79a6b3111216` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:46.581913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=094cab1f24584e54965c79a6b3111216)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:46.583893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:46.802218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094cab1f24584e54965c79a6b3111216'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.810112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094cab1f24584e54965c79a6b3111216'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.811194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094cab1f24584e54965c79a6b3111216'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.827259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094cab1f24584e54965c79a6b3111216'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.858041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:46.897790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094cab1f24584e54965c79a6b3111216'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:46.922491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 961/1888] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:46 2022] DELETE /identity/v3/users/094cab1f24584e54965c79a6b3111216 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:47.556954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:47.558448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:47.572576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77b65b-84be-4d50-85c9-630d4a084626 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:47.636470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:47.637792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:47.638231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:47.638734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] PATH_INFO: `/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:47.640283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4bf9fe4d2a4c19aacb61f8ef59fda1)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:47.642676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:47.776794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4bf9fe4d2a4c19aacb61f8ef59fda1', 'request_id': 'req-00e6b627-a095-4aa3-8ead-a47be05c013b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:47.777632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:47.779038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:47.779471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:47.793645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd359b4-de69-40f6-ab99-3034d137b3ef admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:47.855715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4bf9fe4d2a4c19aacb61f8ef59fda1', 'request_id': 'req-00e6b627-a095-4aa3-8ead-a47be05c013b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:47.860418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:47.861878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:47.862167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:47.862445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] PATH_INFO: `/v3/users/09e4625160b34514b287a878885ab9da` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:47.863698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e4625160b34514b287a878885ab9da)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:47.863928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4bf9fe4d2a4c19aacb61f8ef59fda1', 'request_id': 'req-00e6b627-a095-4aa3-8ead-a47be05c013b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:47.865631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:47.875086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] The token cache is being invalidate because project 0c4bf9fe4d2a4c19aacb61f8ef59fda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:47.875377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:47.876217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-00e6b627-a095-4aa3-8ead-a47be05c013b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:47.879361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 928/1889] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:47 2022] DELETE /identity/v3/projects/0c4bf9fe4d2a4c19aacb61f8ef59fda1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:47.884279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453153-cb20-4a39-b0ea-55325894d5c6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:47.884651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453153-cb20-4a39-b0ea-55325894d5c6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:47.885013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453153-cb20-4a39-b0ea-55325894d5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:47.905362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-37453153-cb20-4a39-b0ea-55325894d5c6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:47.906749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37453153-cb20-4a39-b0ea-55325894d5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:47.916670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 929/1890] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:47.925532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:47.929846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:47.929846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:47.947465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:47.948969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:47.950296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:47.962929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 930/1891] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:47.964495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:47.976052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67b50-6a94-42a0-9595-ed445b72a5ed None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:47.981006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:47.981810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:47.982165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:47.982519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:47.983356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:47.984085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:47.986921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.003165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 931/1892] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:55:48.004226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.019119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4681-0dfb-4df9-8b08-f9115b710a4d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.026475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.037477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.037477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.037477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.037477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.037477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.162712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 932/1893] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:48.168361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.173547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e4625160b34514b287a878885ab9da'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:48.188570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e4625160b34514b287a878885ab9da'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:48.189328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e4625160b34514b287a878885ab9da'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:48.212991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e4625160b34514b287a878885ab9da'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:48.265678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27ac79-51aa-4079-aa54-78d438eebe9d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.271829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.272518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.272805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.273086 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.273794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.275816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.338229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24ed4adc-2817-413c-b86e-c6bd26134e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e4625160b34514b287a878885ab9da'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:48.350063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 962/1894] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:47 2022] DELETE /identity/v3/users/09e4625160b34514b287a878885ab9da => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:48.351827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.352131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.352559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.382006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 933/1895] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:48 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:48.385041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.388363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:48.395286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:48.397427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:48.406716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fada-e36c-423b-bf1f-64cce800748c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.483318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.484002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.484315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.484582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.485342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.487198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.492876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 963/1896] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:48.498673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 934/1897] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:48.509818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.535172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46f3b1-6110-4650-9eb8-efcd423725c0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.553869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.555925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.556330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.556721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.559358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.563012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.563778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.566875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:48.580753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 964/1898] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:48.586313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.601709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55abce-d74d-4a11-87bf-fd515f1a5ee8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.622140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.623089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.623469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.625083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] PATH_INFO: `/v3/projects/1de93115460c4c8289a1c0f945ea05d5/users/68af1c6e79994b328efa0f91c7217aad/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.655609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de93115460c4c8289a1c0f945ea05d5, user_id=68af1c6e79994b328efa0f91c7217aad, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.658754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.663722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d8b7b-a43f-4a90-a45b-a34a99b838f9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:48.667903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:48.668553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.668837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.669120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.679537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:48.680977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b10d-31c5-442e-b3c5-84200f9d8591 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:48.691883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 935/1899] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:48.772652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 965/1900] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:48 2022] PUT /identity/v3/projects/1de93115460c4c8289a1c0f945ea05d5/users/68af1c6e79994b328efa0f91c7217aad/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:48.793854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:48.794413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:48.794801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:48.834687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] MFA Rules not processed for user `68af1c6e79994b328efa0f91c7217aad`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:48.836244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:48.935566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 936/1901] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:48.947547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:48.949186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.016795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf9fdc-473b-4406-a992-18ef06091700 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:49.029142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:49.029142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.029142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.029142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.031925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:49.033984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:49.051031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 966/1902] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:49.076112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:49.170793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee2a66-486c-4c7a-a603-6c410d0615ad None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:49.177372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:49.178401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.178897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.187383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] PATH_INFO: `/v3/users/68af1c6e79994b328efa0f91c7217aad` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.187383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68af1c6e79994b328efa0f91c7217aad)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:49.187383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:49.407185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:49.408884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.426369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725c213-93d0-4fa4-a963-b079a293e4f1 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:49.432222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:49.436509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.436509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.436509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] PATH_INFO: `/v3/projects/df02dcad7ee04acaa239746abbd879af` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.436509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df02dcad7ee04acaa239746abbd879af)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:49.438027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:49.524847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68af1c6e79994b328efa0f91c7217aad'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.549432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68af1c6e79994b328efa0f91c7217aad'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.551003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68af1c6e79994b328efa0f91c7217aad'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.572435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68af1c6e79994b328efa0f91c7217aad'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.623523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df02dcad7ee04acaa239746abbd879af', 'request_id': 'req-95e13631-0c55-4d5b-bc48-7a1d00a32831'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.677016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-71108450-0436-4afe-aec1-b1523a1a30e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68af1c6e79994b328efa0f91c7217aad'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.683862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 937/1903] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:49 2022] DELETE /identity/v3/users/68af1c6e79994b328efa0f91c7217aad => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:49.688675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.688972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.689451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.708770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:49.710238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.771046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 938/1904] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:49.781131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:49.782577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.783859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df02dcad7ee04acaa239746abbd879af', 'request_id': 'req-95e13631-0c55-4d5b-bc48-7a1d00a32831'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.792244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df02dcad7ee04acaa239746abbd879af', 'request_id': 'req-95e13631-0c55-4d5b-bc48-7a1d00a32831'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.806668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] The token cache is being invalidate because project df02dcad7ee04acaa239746abbd879af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:49.806668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.807366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:49.809401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 967/1905] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:49 2022] DELETE /identity/v3/projects/df02dcad7ee04acaa239746abbd879af => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:49.813192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:49.816050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.844012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd701f0-2bc2-4261-9888-865f0fdaeed4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:49.848108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:49.848937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.849295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.849644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.851758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:49.864065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:49.865484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:49.875125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 939/1906] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:49.881165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e13631-0c55-4d5b-bc48-7a1d00a32831 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:49.885951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:49.886861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:49.887149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:49.887428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] PATH_INFO: `/v3/users/da2b97e8457240f1a8c689c8bdcdddd3` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:49.888675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da2b97e8457240f1a8c689c8bdcdddd3)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:49.890630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:50.163521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:50.170830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da2b97e8457240f1a8c689c8bdcdddd3'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.181403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da2b97e8457240f1a8c689c8bdcdddd3'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.182631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da2b97e8457240f1a8c689c8bdcdddd3'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.198797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da2b97e8457240f1a8c689c8bdcdddd3'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.221194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:50.363295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-052b2f38-6267-45cd-b732-d664802fc8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da2b97e8457240f1a8c689c8bdcdddd3'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.370753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 968/1907] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:49 2022] DELETE /identity/v3/users/da2b97e8457240f1a8c689c8bdcdddd3 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:50.590983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:50.592791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:50.611390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aee15d-96ac-488b-a4f4-bf7564881be7 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:50.720741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:50.721624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:50.721912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:50.722192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] PATH_INFO: `/v3/projects/1de93115460c4c8289a1c0f945ea05d5` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:50.723506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de93115460c4c8289a1c0f945ea05d5)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:50.725543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:50.885397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de93115460c4c8289a1c0f945ea05d5', 'request_id': 'req-ff343d83-6454-41ca-ae6c-547f1524dc8b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.953208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de93115460c4c8289a1c0f945ea05d5', 'request_id': 'req-ff343d83-6454-41ca-ae6c-547f1524dc8b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.963280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de93115460c4c8289a1c0f945ea05d5', 'request_id': 'req-ff343d83-6454-41ca-ae6c-547f1524dc8b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.990499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] The token cache is being invalidate because project 1de93115460c4c8289a1c0f945ea05d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:50.991202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:50.994272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ff343d83-6454-41ca-ae6c-547f1524dc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.000953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 940/1908] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:50 2022] DELETE /identity/v3/projects/1de93115460c4c8289a1c0f945ea05d5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:51.006346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.006720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.007024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.062859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:51.065082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.138607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 969/1909] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:51.142935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.143310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.143733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.167930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.172401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.174439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:51.176085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.188286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 941/1910] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:51.194914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.196746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacabf7-cf64-431b-9978-af0b88cc240d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.201871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.202773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.203059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.203497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] PATH_INFO: `/v3/projects/88b833508eac4a4caaef4a57b12c333c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.204904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b833508eac4a4caaef4a57b12c333c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.207452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.209142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baafd5-d8e7-4ca7-9c62-736d499b7434 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.215131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.215915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.216471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.216888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.217746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:51.218281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.220167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.233607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 942/1911] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:51.238167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.248739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2e354-a554-4d68-9d17-fbe845d758aa admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.254300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.255009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.255299 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.255572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.256310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.258267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.325042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b833508eac4a4caaef4a57b12c333c', 'request_id': 'req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.416132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 943/1912] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:51.420189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.436141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b833508eac4a4caaef4a57b12c333c', 'request_id': 'req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.443722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b833508eac4a4caaef4a57b12c333c', 'request_id': 'req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.456627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] The token cache is being invalidate because project 88b833508eac4a4caaef4a57b12c333c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:51.456922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.457614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-acf3ee8b-fd3a-4352-811b-7cedcb5bf321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:51.459259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 970/1913] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:51 2022] DELETE /identity/v3/projects/88b833508eac4a4caaef4a57b12c333c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:51.465755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0382-fbee-4c58-b678-bc497aa76f67 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.466128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0382-fbee-4c58-b678-bc497aa76f67 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.466490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0382-fbee-4c58-b678-bc497aa76f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.485836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-cefe0382-fbee-4c58-b678-bc497aa76f67 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:51.487575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cefe0382-fbee-4c58-b678-bc497aa76f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.505229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ddd2-4acc-4d4f-8b58-2af358338b2a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.510791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.511538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.511954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.512417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.513187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.515840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.550430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 971/1914] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:51.560298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.560586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.560864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.581322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:51.583035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.599406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 972/1915] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:51.606619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 944/1916] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:51.607167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.607843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.611750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.625865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0a11-0cea-4b87-a609-406468ff7cf4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.631513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed06b7-cd61-4fb2-b5f9-70b5cfbbe1e6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.635440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.636157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.636469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.636736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.637229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.637491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:51.637942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.638261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.638615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.638986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.639436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.645610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.647527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.664538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 973/1917] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:51.664928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 945/1918] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:51.668260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.668472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.680001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73323-ab5c-472a-aff1-db4dc7dde84d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.681376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86d34-9b4d-4ce6-9765-143ac8f941ac admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.685844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.687000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.687575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.687841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.688307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.688497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.688950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.689485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.690095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.691140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.692731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.693158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.711248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 946/1919] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:51.712154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.725184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014d180-4b0f-4289-a608-a59cce29633b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:51.732197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:51.733331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.733814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.734426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] PATH_INFO: `/v3/projects/b995882e6eaf4d2daf2cd027c1192d4f/users/167bc621895c4cfeacc9af7791a95aa9/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.752510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b995882e6eaf4d2daf2cd027c1192d4f, user_id=167bc621895c4cfeacc9af7791a95aa9, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:51.755357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e8915-29a1-4814-ad97-2f2cfbe64877 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:51.813124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 974/1920] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:51.813124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:51.841032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 947/1921] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:51 2022] PUT /identity/v3/projects/b995882e6eaf4d2daf2cd027c1192d4f/users/167bc621895c4cfeacc9af7791a95aa9/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:51.851995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:51.852534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:51.852966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:51.887486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] MFA Rules not processed for user `167bc621895c4cfeacc9af7791a95aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:51.888967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:51.931895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc306e-3d59-4474-958e-d22e390a9a01 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.025314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 948/1922] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:52.039976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.040904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.041199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.041497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.042211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.045063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.045201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:52.055636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.057301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:52.082576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:52.172397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b45f60-a600-46b6-9c24-94a2343ea95f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.178005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.178712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.179002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.179280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.206741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.208342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.218064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 975/1923] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:51 2022] POST /identity/v3/users => generated 286 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:52.220238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.225154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 949/1924] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:52.243375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbf38e-31a9-495f-8563-e9b9bebd1cc5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.252377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.253171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.253549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.253919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.254785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.256657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.299256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 976/1925] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:52.304236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.323346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f03d-118c-47ad-a1ae-0abd5dc17ede service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.340026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.342087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.342546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.343035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.348783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.348783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139e9dd-b367-4ef5-b316-da8b5e82dc7b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.360597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 950/1926] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:52.365178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.381643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ec4b-98f9-491c-9241-198e1d2f018f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.387098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.388320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.388759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.389191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] PATH_INFO: `/v3/projects/344845c572e24cf3b86cbda09dc105de/users/71f1fc4adf014408bdb89d183edd63c0/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.406319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=344845c572e24cf3b86cbda09dc105de, user_id=71f1fc4adf014408bdb89d183edd63c0, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.409418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.516369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 977/1927] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:52 2022] PUT /identity/v3/projects/344845c572e24cf3b86cbda09dc105de/users/71f1fc4adf014408bdb89d183edd63c0/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:52.528828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.529118 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.529400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.573753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] MFA Rules not processed for user `71f1fc4adf014408bdb89d183edd63c0`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:52.575327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:52.665814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 951/1928] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:52.679325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.681269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:52.742908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7484d-e84c-4241-9568-cbae0ac23abf admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.747829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.748486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.748948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.749227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.759394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.760941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dab88-6978-4547-94fd-baeabbb6268d service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.771217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 978/1929] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:52.792074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:52.858099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c5426-c018-465b-9161-2a451e8be087 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:52.863268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:52.864338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:52.864530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:52.864797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] PATH_INFO: `/v3/users/71f1fc4adf014408bdb89d183edd63c0` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:52.866036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f1fc4adf014408bdb89d183edd63c0)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:52.868043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:52.878257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:53.066231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f1fc4adf014408bdb89d183edd63c0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:53.073761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f1fc4adf014408bdb89d183edd63c0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:53.074526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f1fc4adf014408bdb89d183edd63c0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:53.089786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f1fc4adf014408bdb89d183edd63c0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:53.168382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f1fc4adf014408bdb89d183edd63c0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:53.175396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 952/1930] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:52 2022] DELETE /identity/v3/users/71f1fc4adf014408bdb89d183edd63c0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:53.179996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:53.180748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:53.181036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:53.203603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:53.205203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:53.286802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 979/1931] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:53.319326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:53.320769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:53.402884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe275d-de83-41c0-923b-df95f0ea0658 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:53.406554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:53.407335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:53.407622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:53.407906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:53.417869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:53.419456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05525852-ca22-410f-bf32-668c1f7ca6a1 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:53.429546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 953/1932] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:53.558059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:53.905956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:54.006962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.008662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:54.038780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b901b-49a9-470a-a561-b4276ee16eeb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.040716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.042352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.043573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.043934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] PATH_INFO: `/v3/projects/344845c572e24cf3b86cbda09dc105de` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.047568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344845c572e24cf3b86cbda09dc105de)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.053792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.177907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344845c572e24cf3b86cbda09dc105de', 'request_id': 'req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:54.248442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344845c572e24cf3b86cbda09dc105de', 'request_id': 'req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:54.256606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344845c572e24cf3b86cbda09dc105de', 'request_id': 'req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:54.269638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] The token cache is being invalidate because project 344845c572e24cf3b86cbda09dc105de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:54.269925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:54.270588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f4bab5c-fdd8-4b51-a803-71069cbbb4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:54.275738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 980/1933] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:54 2022] DELETE /identity/v3/projects/344845c572e24cf3b86cbda09dc105de => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:54.277740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.278055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.278341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.297911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:54.299729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:54.388516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 954/1934] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:54.398830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.399278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.399480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.426327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:54.428414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:54.437698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 981/1935] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:54.442173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.452201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd3e3-a34b-4548-894e-25473d11fcef None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.458316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.459283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.459768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.460235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.462304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:54.462809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.465024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.466698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:54.488361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 955/1936] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:54.492814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.503144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e661ca-65a1-479c-98fe-9e955fe6ff79 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.508678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.509738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.510211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.510875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.515785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.515785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.624335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 982/1937] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:54 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:54.626199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.636938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323c8f6-6378-43c3-85a2-cca19955b814 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.703645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.704420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.704831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.705201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.706119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.709031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.824024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 956/1938] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:54 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:54.828409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.839443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdf8a7-f87e-45dc-a137-1e69cd23cbce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.844088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.844851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.845137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.846099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.846864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.848474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.858428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 983/1939] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:54.862796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.873396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f74d9-e8b3-41a6-a0e8-8b7b291f3bbe admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.878469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.879101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.879388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.879668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.880408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.882073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba3f38-808e-4a49-b1e7-5075aaa17aa9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:54.895511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 957/1940] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:54.899822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:54.910900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd6c48-61c9-4516-946d-189355c268b8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:54.915887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:54.916852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:54.917175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:54.917458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] PATH_INFO: `/v3/projects/9b94572f916546f5a40d2d2f8375bdc0/users/51db4fc011164fed86ba2ae44b85584d/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:54.939545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b94572f916546f5a40d2d2f8375bdc0, user_id=51db4fc011164fed86ba2ae44b85584d, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:54.940992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:55.037327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 984/1941] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:54 2022] PUT /identity/v3/projects/9b94572f916546f5a40d2d2f8375bdc0/users/51db4fc011164fed86ba2ae44b85584d/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:55.047545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:55.048101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:55.048177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:55.080962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] MFA Rules not processed for user `51db4fc011164fed86ba2ae44b85584d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:55.082198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:55.207117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 958/1942] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:55.274216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:55.276003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:55.341846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eee05-853e-46d4-af1d-8e595ccdc452 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:55.352150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:55.352150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:55.352150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:55.352150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:55.358528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:55.360021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869660d8-a66e-4da0-b309-addc23f659ec service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:55.370366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 985/1943] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:55.624053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:55.625580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:55.733418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:55.757990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660533-4dfe-4fc2-99d5-b8d3a4ff9e39 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:55.770954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:55.771970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:55.772384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:55.772720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] PATH_INFO: `/v3/users/8a666d23da3342b099202779665a3e50` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:55.775445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a666d23da3342b099202779665a3e50)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:55.776599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:55.934409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:56.022952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a666d23da3342b099202779665a3e50'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:56.030948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a666d23da3342b099202779665a3e50'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:56.032489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a666d23da3342b099202779665a3e50'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:56.045708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a666d23da3342b099202779665a3e50'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:56.126657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a666d23da3342b099202779665a3e50'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:56.137183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 959/1944] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:55 2022] DELETE /identity/v3/users/8a666d23da3342b099202779665a3e50 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:56.141811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.142213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.142659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.170913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.170913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.260272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 986/1945] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:56.278844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:56.280336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.302688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:56.366183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ecf20-b1bb-4fb3-8b28-148d11f3f187 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:56.370470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:56.371262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.371622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.371966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.381933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:56.383423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:56.394389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 960/1946] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:57.093863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.096990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.122090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f49e8-d2c9-455f-9a9c-43d4d9927b18 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.127743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.128718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.128957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.129431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] PATH_INFO: `/v3/projects/01ba559017f1442ea8a636e3c749a4b7` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.130848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ba559017f1442ea8a636e3c749a4b7)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.133895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.279052 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ba559017f1442ea8a636e3c749a4b7', 'request_id': 'req-7d9130db-5edd-4c75-a78b-5e762bfaccd6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.350902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ba559017f1442ea8a636e3c749a4b7', 'request_id': 'req-7d9130db-5edd-4c75-a78b-5e762bfaccd6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.357790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ba559017f1442ea8a636e3c749a4b7', 'request_id': 'req-7d9130db-5edd-4c75-a78b-5e762bfaccd6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.376442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] The token cache is being invalidate because project 01ba559017f1442ea8a636e3c749a4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:57.377657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.378424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.380451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 987/1947] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:57 2022] DELETE /identity/v3/projects/01ba559017f1442ea8a636e3c749a4b7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:57.384681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.396520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ac481-3824-4221-9a4f-fa8e48bae268 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.398790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:57.460183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.461292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.461730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.462158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] PATH_INFO: `/v3/users/e1b9b35da028475ca032e657634f3400` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.463746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b9b35da028475ca032e657634f3400)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.465698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.667815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b9b35da028475ca032e657634f3400'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.675908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b9b35da028475ca032e657634f3400'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.676658 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b9b35da028475ca032e657634f3400'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.691779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b9b35da028475ca032e657634f3400'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.771778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b9b35da028475ca032e657634f3400'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:57.778391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 961/1948] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:57 2022] DELETE /identity/v3/users/e1b9b35da028475ca032e657634f3400 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:55:57.943471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.338344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.340575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.356251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9130db-5edd-4c75-a78b-5e762bfaccd6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.412019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.451350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.452356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.452640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.454017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] PATH_INFO: `/v3/projects/0869acb011fe4aa1a46394751d41ae5f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.455375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0869acb011fe4aa1a46394751d41ae5f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.457505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.611314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0869acb011fe4aa1a46394751d41ae5f', 'request_id': 'req-50341c85-76dc-4af0-a2cd-7ae69ad881af'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:58.666356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.670298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.684797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90df71e-f3e5-4ab4-8934-c1f49f31a3fa admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.692451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0869acb011fe4aa1a46394751d41ae5f', 'request_id': 'req-50341c85-76dc-4af0-a2cd-7ae69ad881af'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:58.702470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0869acb011fe4aa1a46394751d41ae5f', 'request_id': 'req-50341c85-76dc-4af0-a2cd-7ae69ad881af'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:58.717094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] The token cache is being invalidate because project 0869acb011fe4aa1a46394751d41ae5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:55:58.717722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:58.719634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-50341c85-76dc-4af0-a2cd-7ae69ad881af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:58.722045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 988/1949] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:58 2022] DELETE /identity/v3/projects/0869acb011fe4aa1a46394751d41ae5f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.727335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4a6f0-a080-4ff9-9fe4-ed658a2bbd4f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.727900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4a6f0-a080-4ff9-9fe4-ed658a2bbd4f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.728307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4a6f0-a080-4ff9-9fe4-ed658a2bbd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.758593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-a1a4a6f0-a080-4ff9-9fe4-ed658a2bbd4f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:58.760376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a1a4a6f0-a080-4ff9-9fe4-ed658a2bbd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.791131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.792019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.792309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.792775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] PATH_INFO: `/v3/users/167bc621895c4cfeacc9af7791a95aa9` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.794038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=167bc621895c4cfeacc9af7791a95aa9)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.796251 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.837650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 989/1950] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:58.846448 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.846830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.847113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.867739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:58.869017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.877193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 990/1951] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:58.882549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.897373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba183a1e-0259-458d-a43c-5b4261f5c190 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.902489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.903434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.903829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.904191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.904917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:58.905230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.907597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.924197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 991/1952] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:55:58.928409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.943705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ddf3a-cf7e-4642-b49d-c704107934a7 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.948877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.949548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.949968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.950257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.951017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.952971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.062729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 992/1953] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:55:59.067442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.071379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167bc621895c4cfeacc9af7791a95aa9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:59.079332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167bc621895c4cfeacc9af7791a95aa9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:59.081942 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167bc621895c4cfeacc9af7791a95aa9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:59.092409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.094861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167bc621895c4cfeacc9af7791a95aa9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:59.131244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568a8e-5ce1-4b6c-b71c-8e46af22917f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.136210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.136881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.137167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.137447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.138155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.140193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.189659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1a5d266b-6e96-4a7a-9203-20be4887f65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167bc621895c4cfeacc9af7791a95aa9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:55:59.190147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.197288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 962/1954] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:55:58 2022] DELETE /identity/v3/users/167bc621895c4cfeacc9af7791a95aa9 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:59.202768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.202768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.203188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.221592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.222843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.271965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 993/1955] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:59.275903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.277626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.282924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 963/1956] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:55:59.289901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c7cde-1369-424f-a115-33f2a5b481ff admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.293270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.294746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.295237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.296090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.296551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.296875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.297611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.300002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.310963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 994/1957] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:59.316041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.326412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a27d8-2f07-4b35-842a-6a09fa7ea88e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.331265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.331971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.332294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.332576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.334733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.335712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.346678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 995/1958] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:55:59.351060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.356777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b93cb5-79e3-4741-8c73-9bab7c12ff6f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.360477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449607b1-c948-4405-9360-44ea95575933 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.361414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.361843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.362580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.362907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.365674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.366526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.366841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.367119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] PATH_INFO: `/v3/projects/7741ee0164ad40b2815f7a3ff82b3585/users/863c2646d2a547bab3175fa8f92eb89b/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.372569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.374005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5bac6-bdca-4d06-a37c-65888ba86692 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.383373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7741ee0164ad40b2815f7a3ff82b3585, user_id=863c2646d2a547bab3175fa8f92eb89b, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.384229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 964/1959] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:55:59.386371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.486693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 996/1960] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:55:59 2022] PUT /identity/v3/projects/7741ee0164ad40b2815f7a3ff82b3585/users/863c2646d2a547bab3175fa8f92eb89b/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:59.498340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.498647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.498950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.528501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] MFA Rules not processed for user `863c2646d2a547bab3175fa8f92eb89b`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.529700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.598788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 965/1961] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.613561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.709685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec1f93-350c-4096-8575-ee9fd3e22078 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.714520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.715290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.715647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.716024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.716792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.718925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.826317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.826988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 997/1962] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/projects => generated 389 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:59.828894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.830414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.875879 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fefd7-b3be-4fe9-9fa7-383f0bb92b0b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.893553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.896533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.911589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318909-ab3b-4932-958d-285edde14d78 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.916631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.917310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.917598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.917901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.918596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.920668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.948261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.949221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.949607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.949945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] PATH_INFO: `/v3/projects/b995882e6eaf4d2daf2cd027c1192d4f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.951238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b995882e6eaf4d2daf2cd027c1192d4f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.953812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.036404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 998/1963] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/users => generated 301 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:00.040375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.051158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223cc16-a97f-48aa-a177-10a9e4114319 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.056137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.056844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.057163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.057503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.058438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.060182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.071957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 999/1964] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:00.076772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.090152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe236fc-e1f7-4b0a-a415-ba93465ba322 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.101566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b995882e6eaf4d2daf2cd027c1192d4f', 'request_id': 'req-6b582a38-f13d-434a-8dfe-c31f716fcc06'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:00.147657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.148478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.148752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.149029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] PATH_INFO: `/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe/users/52549742277848289bbd920bda048160/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.164313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dc3e913f9b473094cab4eae48b0ebe, user_id=52549742277848289bbd920bda048160, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.166206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b995882e6eaf4d2daf2cd027c1192d4f', 'request_id': 'req-6b582a38-f13d-434a-8dfe-c31f716fcc06'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:00.168404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.173179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b995882e6eaf4d2daf2cd027c1192d4f', 'request_id': 'req-6b582a38-f13d-434a-8dfe-c31f716fcc06'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:00.187780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] The token cache is being invalidate because project b995882e6eaf4d2daf2cd027c1192d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:00.188109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:00.188914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b582a38-f13d-434a-8dfe-c31f716fcc06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:00.191378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 966/1965] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:55:59 2022] DELETE /identity/v3/projects/b995882e6eaf4d2daf2cd027c1192d4f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:00.196288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6386-73e2-40a4-8bfa-4df6d4d737d9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.196605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6386-73e2-40a4-8bfa-4df6d4d737d9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.196895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6386-73e2-40a4-8bfa-4df6d4d737d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.215561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-93be6386-73e2-40a4-8bfa-4df6d4d737d9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.216778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-93be6386-73e2-40a4-8bfa-4df6d4d737d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.225801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 967/1966] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:00.235587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.235912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.236228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.254911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.256128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.264857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 968/1967] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:00.269651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.276569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1000/1968] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:00 2022] PUT /identity/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe/users/52549742277848289bbd920bda048160/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:00.279110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17995fdd-0956-4fd5-8f90-d556a7fbaf2a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.281050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.283454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.346376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.347078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.347364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.347642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.348372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:00.348682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.350432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.353692 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bfaf-8e0f-4378-8ee5-d6a9e2665dc5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.358791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.359633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.359953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.360238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.361089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.362317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 969/1969] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:00.362864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.367083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.374675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1001/1970] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:00.377535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.377756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2ab99-9bcf-48cd-9fd6-374f238c482e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.382755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.383410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.383743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.384019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.384755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.386704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.387828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437932f-d9bb-41e1-8371-5238cbc0c45c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.393080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.393969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.394403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.394711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] PATH_INFO: `/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe/users/52549742277848289bbd920bda048160/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.396681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dc3e913f9b473094cab4eae48b0ebe, user_id=52549742277848289bbd920bda048160, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.399405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567de49-35bd-4c26-b390-a6b751a5d947 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.410755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1002/1971] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:00 2022] PUT /identity/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe/users/52549742277848289bbd920bda048160/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:00.426187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.426481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.426809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.453634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] MFA Rules not processed for user `52549742277848289bbd920bda048160`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.455524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.475037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 970/1972] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:00.475210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:00.476096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.485935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:00.525286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1003/1973] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:00.536510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.537997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e5ffc-0e7f-45df-a60d-55c06d9c5563 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.543118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.543229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.543507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.543792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.544065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.544754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.546670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.595736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e52cb-0717-4a6e-8ec6-eab7fd6a50d6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.650904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 971/1974] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:00.651084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.651468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.651827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.652221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.652409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.667350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b53780-b638-48b5-af80-b23ca13c9ba5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.677050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.677050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.677420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.677678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.678895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.681538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.704377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 972/1975] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:00.710112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.727707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe49d14-e105-4c53-b061-7b89aedbefda admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.735629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.736703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.737141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.737654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.739058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.741871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.750618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.754612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.765810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 973/1976] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:00.767827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.771259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1004/1977] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:00.779835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fbe23-7c28-4000-ad9e-31387f569b9e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.786247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.787173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.787462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.787740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] PATH_INFO: `/v3/projects/ffc7e400990f44a4af1367e01ef0cd1e/users/1317239afd014696b93d14838fda287b/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.804380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc7e400990f44a4af1367e01ef0cd1e, user_id=1317239afd014696b93d14838fda287b, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.807199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291f584-f564-4300-8698-e22723552eb4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.910502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.921155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d2ac-2ba2-4593-8c60-aaeb1513b459 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.921309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 974/1978] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:00 2022] PUT /identity/v3/projects/ffc7e400990f44a4af1367e01ef0cd1e/users/1317239afd014696b93d14838fda287b/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:00.932811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.933105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.933382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.943660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:00.965122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] MFA Rules not processed for user `1317239afd014696b93d14838fda287b`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.966428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.992088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.992088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.992433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.992791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] PATH_INFO: `/v3/users/863c2646d2a547bab3175fa8f92eb89b` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.994180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863c2646d2a547bab3175fa8f92eb89b)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.996591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.056055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 975/1979] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:01.069436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:01.071320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:01.122220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:01.134328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414f640-2cf0-4095-ae43-7d773e6e7836 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:01.138034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:01.138712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.139003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.139286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.148195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:01.149642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44145b-b5e0-40d5-98e4-ec20c5a142d8 service glance] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.159556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 976/1980] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:01.247504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863c2646d2a547bab3175fa8f92eb89b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:01.255377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863c2646d2a547bab3175fa8f92eb89b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:01.256139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863c2646d2a547bab3175fa8f92eb89b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:01.269622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863c2646d2a547bab3175fa8f92eb89b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:01.345097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863c2646d2a547bab3175fa8f92eb89b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:01.352329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1005/1981] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:56:00 2022] DELETE /identity/v3/users/863c2646d2a547bab3175fa8f92eb89b => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:01.359018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.359018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.359018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.375477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:01.376789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:01.446576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 977/1982] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:01.457280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:01.458811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:01.515584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d9709-af90-4b29-a227-e7b7de88b112 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:01.519171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:01.519816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.520102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.520379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.529417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:01.530902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.540978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1006/1983] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:02.064579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:02.067283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:02.086205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d44a9-6dac-4744-88c5-11d80533c561 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:02.093227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26460517-072e-4662-b71e-981c35252620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:02.093844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26460517-072e-4662-b71e-981c35252620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.093844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26460517-072e-4662-b71e-981c35252620 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.094139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26460517-072e-4662-b71e-981c35252620 admin admin] PATH_INFO: `/v3/projects/7741ee0164ad40b2815f7a3ff82b3585` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.095797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26460517-072e-4662-b71e-981c35252620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7741ee0164ad40b2815f7a3ff82b3585)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:02.098305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26460517-072e-4662-b71e-981c35252620 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:02.223475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7741ee0164ad40b2815f7a3ff82b3585', 'request_id': 'req-26460517-072e-4662-b71e-981c35252620'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.320089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7741ee0164ad40b2815f7a3ff82b3585', 'request_id': 'req-26460517-072e-4662-b71e-981c35252620'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.334078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7741ee0164ad40b2815f7a3ff82b3585', 'request_id': 'req-26460517-072e-4662-b71e-981c35252620'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.339271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] The token cache is being invalidate because project 7741ee0164ad40b2815f7a3ff82b3585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:02.339559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.340258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.341848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 978/1984] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:56:02 2022] DELETE /identity/v3/projects/7741ee0164ad40b2815f7a3ff82b3585 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:02.346118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:02.355941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ab23b-b6fd-4220-868a-138602d3152b service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:02.401454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:02.417051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:02.417923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.418229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.418510 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] PATH_INFO: `/v3/users/52549742277848289bbd920bda048160` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.419748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52549742277848289bbd920bda048160)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:02.421999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:02.679348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52549742277848289bbd920bda048160'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.686160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52549742277848289bbd920bda048160'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.686858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52549742277848289bbd920bda048160'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.699409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52549742277848289bbd920bda048160'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.813744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-408236df-50ba-4628-81ae-9a0ca3976a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52549742277848289bbd920bda048160'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:02.820858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1007/1985] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:56:02 2022] DELETE /identity/v3/users/52549742277848289bbd920bda048160 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:03.273355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.275032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:03.292431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26460517-072e-4662-b71e-981c35252620 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.353895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.354807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.355097 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.355380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] PATH_INFO: `/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.356594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8dc3e913f9b473094cab4eae48b0ebe)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.358519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.369524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:03.530395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8dc3e913f9b473094cab4eae48b0ebe', 'request_id': 'req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:03.618745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8dc3e913f9b473094cab4eae48b0ebe', 'request_id': 'req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:03.625766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8dc3e913f9b473094cab4eae48b0ebe', 'request_id': 'req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:03.636696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] The token cache is being invalidate because project b8dc3e913f9b473094cab4eae48b0ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:03.636986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:03.637624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cec725b1-fbeb-4dfc-8558-63a29ffc2a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:03.639213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 979/1986] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:56:03 2022] DELETE /identity/v3/projects/b8dc3e913f9b473094cab4eae48b0ebe => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:03.644676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.644968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.645250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.663351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:03.664559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:03.724093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1008/1987] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:03.732181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.732465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.732766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.751819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:03.753070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:03.761970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 980/1988] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:03.766804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.778111 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0fa47-315f-4ee9-afc6-ab988f629622 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.783112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.783704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.783987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.784264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.784990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:03.785302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.787006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.798693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1009/1989] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:03.803785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.813542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e149577-f794-4f23-bfcf-292b34b9ef3e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.819356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.820023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.820307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.820581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.821433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.823674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.924954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 981/1990] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:03.930065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.942570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b74630-73fa-4806-801e-fdb44db049e0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.988255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:03.998211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.998922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.999215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.999502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.003819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.003819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.087892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1010/1991] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:04.094867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.106824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d217f78-157d-4c0f-9150-1f5787eed3d9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.111997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.112714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.114747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.114747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.114747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.116672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.127406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 982/1992] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:04.131398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.141414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027accc2-588a-45bb-b35b-f37454baca3f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.146090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.146788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.147083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.149216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.149216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.149744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2f6b1-1974-47e0-8efc-73b24d07491a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.164123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1011/1993] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:04.165602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.178007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a9df-3ffb-4d21-8332-525da0eefbc1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.182343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.183640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.183989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.184373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] PATH_INFO: `/v3/projects/f3fe34cb175740b299804b8c9d2f3371/users/7792f354950d41a3b0b81267cd2714cc/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.206685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3fe34cb175740b299804b8c9d2f3371, user_id=7792f354950d41a3b0b81267cd2714cc, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.209379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.334226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 983/1994] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:04 2022] PUT /identity/v3/projects/f3fe34cb175740b299804b8c9d2f3371/users/7792f354950d41a3b0b81267cd2714cc/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:04.339563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe07d67-135a-460b-98a1-bb06c69383ae None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.339844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe07d67-135a-460b-98a1-bb06c69383ae None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.340186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe07d67-135a-460b-98a1-bb06c69383ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.359783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5fe07d67-135a-460b-98a1-bb06c69383ae None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:04.361262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5fe07d67-135a-460b-98a1-bb06c69383ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:04.424016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1012/1995] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:04.434013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.436127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:04.492683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5ec5-45c1-46b0-8c57-b4261ec78ce4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.496428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.497065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.497353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.497635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.506679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.508139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.517690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 984/1996] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:06.254791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82ce2-f679-4b25-b678-71b10c3b0f77 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.255631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82ce2-f679-4b25-b678-71b10c3b0f77 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.255631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82ce2-f679-4b25-b678-71b10c3b0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.307362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3cc82ce2-f679-4b25-b678-71b10c3b0f77 None None] MFA Rules not processed for user `7792f354950d41a3b0b81267cd2714cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:06.308709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3cc82ce2-f679-4b25-b678-71b10c3b0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:06.421705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1013/1997] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:06.433361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:06.434872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:06.478845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:06.496217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d5377-ef6a-4783-8259-4b0e8c4f5b37 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:06.502063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:06.503455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.503628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.504814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.515084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:06.518564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:06.528947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 985/1998] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:09.411931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6af3-a10d-4a08-8570-dce04a3dd299 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.412508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6af3-a10d-4a08-8570-dce04a3dd299 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.412726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6af3-a10d-4a08-8570-dce04a3dd299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.431446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4e2b6af3-a10d-4a08-8570-dce04a3dd299 None None] MFA Rules not processed for user `7792f354950d41a3b0b81267cd2714cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:09.432738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4e2b6af3-a10d-4a08-8570-dce04a3dd299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:09.439788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1014/1999] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:10.232395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:10.233910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:10.309010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dd12-6937-4d2e-946f-01f523cc6c51 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.321637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.322719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.323202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.323441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] PATH_INFO: `/v3/users/51db4fc011164fed86ba2ae44b85584d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.324925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51db4fc011164fed86ba2ae44b85584d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.327037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.547967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51db4fc011164fed86ba2ae44b85584d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:10.559402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51db4fc011164fed86ba2ae44b85584d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:10.560165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51db4fc011164fed86ba2ae44b85584d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:10.574515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51db4fc011164fed86ba2ae44b85584d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:10.635400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51db4fc011164fed86ba2ae44b85584d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:10.643837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 986/2000] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:56:10 2022] DELETE /identity/v3/users/51db4fc011164fed86ba2ae44b85584d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:10.648920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.649590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.650717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.671135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:10.672573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:10.732095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:10.747584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1015/2001] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:10.760625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:10.762268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:10.852773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:10.870210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6574-79e7-4566-9ae7-fc6e3edf7c21 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.874183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.878289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.878289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.878289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.885420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.886904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397efc-16e9-479d-beae-5c06e5cc258c service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.897973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 987/2002] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:11.580830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.582479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:11.596994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0511d6-f2f9-4ffe-bda4-98f1b83e1f6c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.603202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.604148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.604629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.605094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] PATH_INFO: `/v3/projects/9b94572f916546f5a40d2d2f8375bdc0` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.606741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b94572f916546f5a40d2d2f8375bdc0)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.608991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.776054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b94572f916546f5a40d2d2f8375bdc0', 'request_id': 'req-f30d8e05-7a22-4191-ad3b-506540a92c69'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:11.862057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b94572f916546f5a40d2d2f8375bdc0', 'request_id': 'req-f30d8e05-7a22-4191-ad3b-506540a92c69'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:11.869856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b94572f916546f5a40d2d2f8375bdc0', 'request_id': 'req-f30d8e05-7a22-4191-ad3b-506540a92c69'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:11.881529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] The token cache is being invalidate because project 9b94572f916546f5a40d2d2f8375bdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:11.882058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:11.882802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f30d8e05-7a22-4191-ad3b-506540a92c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:11.884985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1016/2003] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:56:11 2022] DELETE /identity/v3/projects/9b94572f916546f5a40d2d2f8375bdc0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:11.890602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.891182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.891445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.913935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:11.915286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:11.999179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 988/2004] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:12.010875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.011175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.013587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.032567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:12.033935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.055115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1017/2005] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:12.063632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.080514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995e973-2acc-44fd-99c6-186ac9aab5f7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.086020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.086849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.087206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.087715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.088530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:12.088906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.090712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.109420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 989/2006] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:12.110609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.121959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918410b-4c50-4175-8cc6-34488e72998e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.127429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.128307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.128706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.129092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.130118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.133871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.230323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1018/2007] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:56:12 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:12.232424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.243950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c46-4c23-4e8b-b187-7ab852eca7b2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.319725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.320412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.320696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.321012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.321684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.323911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.457127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 990/2008] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:56:12 2022] POST /identity/v3/users => generated 284 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:12.461730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.474032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e38a9-4593-4b26-af83-ee10c274b778 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.477939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.478881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.479186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.479458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.480205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.482084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.494916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1019/2009] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:12.499287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.514590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f9cd0-7ddc-4f25-8ff8-083815902356 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.521325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.522134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.523060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.523352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.524318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.525836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f3438-1da4-41e2-97de-1a6e278449df admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.537286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 991/2010] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:12.541711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.552530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc410-1fd5-407f-bcc6-8b15daab5c04 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.557563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.558592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.558861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.559287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] PATH_INFO: `/v3/projects/cff11022a88846d8a80c1f10dd2b2810/users/810b106f4c8541fe81677395ac013724/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.579086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff11022a88846d8a80c1f10dd2b2810, user_id=810b106f4c8541fe81677395ac013724, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.581909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.715716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1020/2011] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:12 2022] PUT /identity/v3/projects/cff11022a88846d8a80c1f10dd2b2810/users/810b106f4c8541fe81677395ac013724/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:12.728401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.728715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.730177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.775027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] MFA Rules not processed for user `810b106f4c8541fe81677395ac013724`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:12.776384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.830528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:12.868442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 992/2012] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:12.888163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.890307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.974576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507b753-6f8a-4ede-80c1-200e146c100d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.979897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.980707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.981103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.981471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.991965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.993490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:13.008764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1021/2013] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:17.436098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:17.438050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:17.579571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b163358-fd9e-43a8-8bab-0d0bb4c5d806 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:17.587171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:17.589023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.590209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.590721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] PATH_INFO: `/v3/users/8a96ba1833514a7ba5f17409d0781026` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.595015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a96ba1833514a7ba5f17409d0781026)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:17.599393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.853600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a96ba1833514a7ba5f17409d0781026'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.860598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a96ba1833514a7ba5f17409d0781026'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.861256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a96ba1833514a7ba5f17409d0781026'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.874058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a96ba1833514a7ba5f17409d0781026'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.933247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-792c9990-4f9e-4e6c-9232-626c8b636764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a96ba1833514a7ba5f17409d0781026'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.940249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 993/2014] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:56:17 2022] DELETE /identity/v3/users/8a96ba1833514a7ba5f17409d0781026 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:18.260263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.271650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239a5b-c4dc-472d-86e6-7ab8def7b8b4 service glance] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.305126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:18.363580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.364469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.364761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.365198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] PATH_INFO: `/v3/projects/4a1a1173d9a24de2b797fdf1c72f4131` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.367105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1a1173d9a24de2b797fdf1c72f4131)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.369046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.484928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1a1173d9a24de2b797fdf1c72f4131', 'request_id': 'req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.556346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1a1173d9a24de2b797fdf1c72f4131', 'request_id': 'req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.565398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1a1173d9a24de2b797fdf1c72f4131', 'request_id': 'req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.579249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] The token cache is being invalidate because project 4a1a1173d9a24de2b797fdf1c72f4131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:18.579567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.580208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2114aa0b-beaf-4f7e-b1fb-1819887f9da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.581796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1022/2015] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:56:18 2022] DELETE /identity/v3/projects/4a1a1173d9a24de2b797fdf1c72f4131 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:18.587524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.588374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.588374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.636850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:18.638294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:18.660831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:18.731710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 994/2016] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:18.739176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.739404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.739691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.759715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:18.761144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:18.770726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1023/2017] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:18.775107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.785485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34849f4b-4716-4b15-a54e-7deb5d9acac0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.790276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.790979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.791261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.791545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.792279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:18.792593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.794241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.815387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 995/2018] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:56:18.821217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.831617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd4762-1502-497e-98dd-2ee35226f0cd None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.845743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.846411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.846739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.847027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.847774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.849781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.924220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1024/2019] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:18.929259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.940370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ccc0d-601f-49ce-8fdb-cbea4d7e5129 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.999226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.999828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.000115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.000522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.001082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.003044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.111501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 996/2020] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:19.117475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.128930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcacd0f-f60e-4595-a298-2c35f5b2bd0b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.133670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.140246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.140246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.140246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.140246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.140246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.147631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1025/2021] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:19.151661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.163517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5d8f-4f82-48be-b965-90e994afbb4f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.168988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.169630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.169913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.170186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.170957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.172644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba6329-0ac8-4608-ae27-0b89e7685dfc admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.184442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 997/2022] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:19.189384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.199866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacccd50-cfc0-4ab7-a954-2c8d8bf6934e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.204885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.205759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.211650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.211650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] PATH_INFO: `/v3/projects/8aa6307af11843eeaf62ed59ae877463/users/b577a8a0aff647279759105fe2bc7a76/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.223359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa6307af11843eeaf62ed59ae877463, user_id=b577a8a0aff647279759105fe2bc7a76, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.226613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.325031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1026/2023] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:19 2022] PUT /identity/v3/projects/8aa6307af11843eeaf62ed59ae877463/users/b577a8a0aff647279759105fe2bc7a76/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:19.336270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.337730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.337730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.370189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] MFA Rules not processed for user `b577a8a0aff647279759105fe2bc7a76`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:19.371621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:19.470999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 998/2024] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:19.482437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.484203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:19.530708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:19.559841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1436-548a-4bf4-ab80-2cd22f09cbd9 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.566339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.567021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.567309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.567593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.575916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.577564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd298a-8f37-4828-baea-4e7c02ca5f41 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.587054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1027/2025] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:29.467949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:29.627921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=15 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:35.205418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.276369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c8b64-9270-44ce-bfe5-2adc2bfd3ac1 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.281231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.282107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.282395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.282705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] PATH_INFO: `/v3/users/b577a8a0aff647279759105fe2bc7a76` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.283958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b577a8a0aff647279759105fe2bc7a76)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.285865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.557948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b577a8a0aff647279759105fe2bc7a76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.565240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b577a8a0aff647279759105fe2bc7a76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.565908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b577a8a0aff647279759105fe2bc7a76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.578113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b577a8a0aff647279759105fe2bc7a76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.669613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b577a8a0aff647279759105fe2bc7a76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.675960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 999/2026] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:56:35 2022] DELETE /identity/v3/users/b577a8a0aff647279759105fe2bc7a76 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:35.680404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.680885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.680980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.698608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:35.699856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.757262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1028/2027] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:35.780699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.780699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.835340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66564-c2a5-46b1-b7ab-911c472d30ab admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.839234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.839869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.840161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.840426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.849428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.851109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b66b6-cd53-45d0-a76d-57fd0b8883d9 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.860557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1000/2028] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:36.355672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.357109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.370594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e5e1c-c17f-4999-8787-1a2ff11b0f12 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.375333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.376262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.376545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.376824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] PATH_INFO: `/v3/projects/8aa6307af11843eeaf62ed59ae877463` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.378316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa6307af11843eeaf62ed59ae877463)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.380463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.503548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa6307af11843eeaf62ed59ae877463', 'request_id': 'req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:36.595485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa6307af11843eeaf62ed59ae877463', 'request_id': 'req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:36.601617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa6307af11843eeaf62ed59ae877463', 'request_id': 'req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:36.612923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] The token cache is being invalidate because project 8aa6307af11843eeaf62ed59ae877463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:36.613215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:36.613932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e9f0ae55-46c6-4a5f-9004-47ea5576e2c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:36.615549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1029/2029] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:56:36 2022] DELETE /identity/v3/projects/8aa6307af11843eeaf62ed59ae877463 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:36.620527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.620977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.621103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.638955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:36.649721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:36.650953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.709735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1001/2030] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:36.722485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.722916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.723092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.743135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:36.745283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.753996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1030/2031] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:36.758812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.769366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860788-c29a-40f1-8e13-e329a29aeb4a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.774212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.774917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.775204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.775480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.776255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:36.776565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.778190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.789650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1002/2032] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:36.794055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.803992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c77b5-0e31-4a53-9c8c-a52065e61c5d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.808761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.809405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.809698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.809968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.810751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.812683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.903874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1031/2033] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:36.907843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.918417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52280b3-3a54-47fe-aa46-37fbc1a4ec49 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.976286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.976984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.977331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.977693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.978665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.980992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.059888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1003/2034] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:37.061381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.072855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccae1d-1ec8-4aed-b825-e04aa3341492 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.077683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.078336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.078621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.078931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.079658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.081251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.092616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1032/2035] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:37.096510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.108388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f647-f938-4e4d-819b-d156706f8ae2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.113313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.113988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.114728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.114728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.115305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.117011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef360f7c-4be1-4e38-ad20-4f31a6b1db20 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.132076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1004/2036] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:37.133056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.143193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7ff8d-9624-4ea0-8b78-ce24cc00a375 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.148461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.149342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.149624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.149901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] PATH_INFO: `/v3/projects/1e722ef565af448a996e5f8fd92fe1ef/users/99bcc9c844d44f4b8b74e9aa8871428e/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.171951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e722ef565af448a996e5f8fd92fe1ef, user_id=99bcc9c844d44f4b8b74e9aa8871428e, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.174835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.264899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1033/2037] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:56:37 2022] PUT /identity/v3/projects/1e722ef565af448a996e5f8fd92fe1ef/users/99bcc9c844d44f4b8b74e9aa8871428e/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:37.275432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d881c49-2fd2-48cb-918d-b519dd3e6708 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.275728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d881c49-2fd2-48cb-918d-b519dd3e6708 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.276012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d881c49-2fd2-48cb-918d-b519dd3e6708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.302522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0d881c49-2fd2-48cb-918d-b519dd3e6708 None None] MFA Rules not processed for user `99bcc9c844d44f4b8b74e9aa8871428e`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:37.304883 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0d881c49-2fd2-48cb-918d-b519dd3e6708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:37.371241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:37.375007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1005/2038] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:37.385732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.387589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:37.410612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:37.445126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7852f-4dd0-4763-bca9-c374a4d42d84 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.448779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.449420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.449705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.449987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.458605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.460606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.469371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1034/2039] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:45.696462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05c8a7-d4da-4bcb-a2e2-2b31fe7a7f6f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.697032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05c8a7-d4da-4bcb-a2e2-2b31fe7a7f6f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.697179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05c8a7-d4da-4bcb-a2e2-2b31fe7a7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.714939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dd05c8a7-d4da-4bcb-a2e2-2b31fe7a7f6f None None] MFA Rules not processed for user `7792f354950d41a3b0b81267cd2714cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:45.716133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dd05c8a7-d4da-4bcb-a2e2-2b31fe7a7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.778248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1006/2040] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:47.835813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=17 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:55.766839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=25 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:02.688675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:02.752493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c46e4-00cc-41c6-8a93-ef013a1557e4 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:02.757602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:02.758476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.758840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.759049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] PATH_INFO: `/v3/users/99bcc9c844d44f4b8b74e9aa8871428e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.760241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bcc9c844d44f4b8b74e9aa8871428e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:02.762199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:02.906169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bcc9c844d44f4b8b74e9aa8871428e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.913056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bcc9c844d44f4b8b74e9aa8871428e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.913831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bcc9c844d44f4b8b74e9aa8871428e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.931558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bcc9c844d44f4b8b74e9aa8871428e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.983364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bcc9c844d44f4b8b74e9aa8871428e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.989670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1035/2041] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:57:02 2022] DELETE /identity/v3/users/99bcc9c844d44f4b8b74e9aa8871428e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:02.994758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.995220 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.995380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.013464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:03.014856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.080204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1007/2042] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:03.088695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.090235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.150610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1b6df-9b36-4bff-8af5-c8e19dc661ed admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.155009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.155653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.155937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.156215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.166419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.167920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdba981-cf9f-4b4f-a1db-af6e432ef7d8 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.186620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1036/2043] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:03.635734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.637998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.652028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b15b6-680c-4556-82ad-e947b0c91ad7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.657822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.658803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.659389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.659389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] PATH_INFO: `/v3/projects/1e722ef565af448a996e5f8fd92fe1ef` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.674950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e722ef565af448a996e5f8fd92fe1ef)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.677125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.787821 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e722ef565af448a996e5f8fd92fe1ef', 'request_id': 'req-34868446-8246-45a5-be83-03e9e5a73b5a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:03.862795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e722ef565af448a996e5f8fd92fe1ef', 'request_id': 'req-34868446-8246-45a5-be83-03e9e5a73b5a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:03.869305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e722ef565af448a996e5f8fd92fe1ef', 'request_id': 'req-34868446-8246-45a5-be83-03e9e5a73b5a'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:03.879655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] The token cache is being invalidate because project 1e722ef565af448a996e5f8fd92fe1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:03.882114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:03.882114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-34868446-8246-45a5-be83-03e9e5a73b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:03.882296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1008/2044] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:57:03 2022] DELETE /identity/v3/projects/1e722ef565af448a996e5f8fd92fe1ef => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:03.889540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.890018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.890132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.908218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:03.909424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.926677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:03.968329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1037/2045] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:03.976481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.976798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.977081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.995000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:03.996195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:04.003998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1009/2046] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:04.008750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.018526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0a82b-0acd-4b0f-89a5-7c3146f6eb5f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.023686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.024330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.024616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.024895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.025616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:04.025927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.027555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.038776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1038/2047] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:04.042958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.052231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773538-2a32-4db6-ae86-da5eb25d39e4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.056877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.057547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.057834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.058110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.058857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.060742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.133076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1010/2048] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:57:04 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:04.137295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.148124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c8702-c6f4-46d3-b99f-4378a81e2c4e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.205227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.205923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.206196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.206479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.207221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.209895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.303612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1039/2049] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:57:04 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:04.304028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.315338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fb12-534c-418f-8f1f-e72a500c652c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.320944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.321643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.321926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.322201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.323090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.324693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.344479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1011/2050] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:57:04.347474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.357567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84177c48-06eb-4dc0-b1a1-21d8fad425e4 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.362141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.362875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.363182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.363446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.364192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.365779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ea62-1d85-41b7-9aca-bd02fb337a12 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.375627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1040/2051] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:04.379868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.391884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198516-50a9-4412-a850-b9813da53161 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.397270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.398419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.398739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.399018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] PATH_INFO: `/v3/projects/4b4de3aedd894cb7adac85188597fec3/users/236860f76ea443008dc35bc4a5214d48/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.413994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4de3aedd894cb7adac85188597fec3, user_id=236860f76ea443008dc35bc4a5214d48, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.416672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.522596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1012/2052] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:57:04 2022] PUT /identity/v3/projects/4b4de3aedd894cb7adac85188597fec3/users/236860f76ea443008dc35bc4a5214d48/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:04.530230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.530463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.530801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.557942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] MFA Rules not processed for user `236860f76ea443008dc35bc4a5214d48`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:04.559207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:04.621696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1041/2053] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:04.626326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.647663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:04.689691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13ba-13ea-4a9e-b511-44912bc18176 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.700392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.700392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.700392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.700392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.700752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.700752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.778830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1013/2054] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:57:04 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:04.783076 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.794460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dad0e-5f82-40b0-88cd-19e32b9e223a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.808334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:04.855318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.855997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.856284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.856563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.857271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.859283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.028113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1042/2055] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:57:04 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:05.031445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.042241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299720-3ab0-4680-a0cd-8ae553611bf6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.046630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.047306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.047590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.047864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.048585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.050180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.061188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1014/2056] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:05.065615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.075509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ef435-84c7-4c54-8014-74442700cc1b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.080062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.080923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.081204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.081479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] PATH_INFO: `/v3/projects/8e24e9eebf39444f976ac07679bfbc6c/users/0c6851fc9ca54c5abcd40685219a8e8e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.096472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e24e9eebf39444f976ac07679bfbc6c, user_id=0c6851fc9ca54c5abcd40685219a8e8e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.099148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.169788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1043/2057] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:57:05 2022] PUT /identity/v3/projects/8e24e9eebf39444f976ac07679bfbc6c/users/0c6851fc9ca54c5abcd40685219a8e8e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:05.173653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.183523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6811f-2546-4df0-ada5-eb79a7eb93ca admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.239425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.240091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.240376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.240654 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.241376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.243140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95316ac5-2e6c-45a9-818e-767553eaca25 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.252708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1015/2058] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:05.256756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.266866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c9213-9ed3-45e5-a06d-b17a15ded157 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.271484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.272351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.272633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.272910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] PATH_INFO: `/v3/projects/8e24e9eebf39444f976ac07679bfbc6c/users/0c6851fc9ca54c5abcd40685219a8e8e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.275402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e24e9eebf39444f976ac07679bfbc6c, user_id=0c6851fc9ca54c5abcd40685219a8e8e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.278075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.288047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1044/2059] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:57:05 2022] PUT /identity/v3/projects/8e24e9eebf39444f976ac07679bfbc6c/users/0c6851fc9ca54c5abcd40685219a8e8e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:05.299731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a95a-7005-453f-a382-5951406914e9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.299963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a95a-7005-453f-a382-5951406914e9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.300250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a95a-7005-453f-a382-5951406914e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.326950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4951a95a-7005-453f-a382-5951406914e9 None None] MFA Rules not processed for user `0c6851fc9ca54c5abcd40685219a8e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:05.328163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4951a95a-7005-453f-a382-5951406914e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:05.401214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1016/2060] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:05.412613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.414353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:05.500572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28eb3-769e-4b71-a833-94780ff45c65 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.504420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.505135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.505485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.505829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.515718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.517140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.527135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1045/2061] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:05.692631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951a95a-7005-453f-a382-5951406914e9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:05.694240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4951a95a-7005-453f-a382-5951406914e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:05.706829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951a95a-7005-453f-a382-5951406914e9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:05.710419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:05.711208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:05.711501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:05.712341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:05.724334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:05.768312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:05.770431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca74e-749b-4249-9719-312761631ccd service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:05.779777 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1017/2062] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:15.522238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:15.775785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=21 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:22.376498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:22.377914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:22.442852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538184e-5f67-461a-9abe-a5edb961ec0d service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:22.447679 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:22.448545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:22.448832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:22.449164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] PATH_INFO: `/v3/users/810b106f4c8541fe81677395ac013724` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:22.450415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=810b106f4c8541fe81677395ac013724)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:22.452532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:22.734474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '810b106f4c8541fe81677395ac013724'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:22.742985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '810b106f4c8541fe81677395ac013724'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:22.743693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '810b106f4c8541fe81677395ac013724'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:22.756219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '810b106f4c8541fe81677395ac013724'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:22.846055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '810b106f4c8541fe81677395ac013724'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:22.853075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1046/2063] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:57:22 2022] DELETE /identity/v3/users/810b106f4c8541fe81677395ac013724 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:22.858207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:22.858793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:22.859010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:22.880090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:22.881397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:22.953093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1018/2064] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:22.963801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:22.965557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:23.030171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6913a1-2b3c-4ca5-aff4-a392b3cab8ab admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:23.040913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:23.040913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:23.041139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:23.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:23.051596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:23.053267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af013f58-acaa-4397-9614-95f9d0f2e90b service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:23.063156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1047/2065] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:57:23.579558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:23.581044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:23.602041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ffde-19fe-4be1-821f-1f1a7c5a4f75 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:23.608858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:23.609770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:23.610154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:23.610346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] PATH_INFO: `/v3/projects/cff11022a88846d8a80c1f10dd2b2810` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:23.611602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff11022a88846d8a80c1f10dd2b2810)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:23.613532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:23.716693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff11022a88846d8a80c1f10dd2b2810', 'request_id': 'req-5d57549b-506b-40d9-b353-a0a38741536f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:23.771242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff11022a88846d8a80c1f10dd2b2810', 'request_id': 'req-5d57549b-506b-40d9-b353-a0a38741536f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:23.778565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff11022a88846d8a80c1f10dd2b2810', 'request_id': 'req-5d57549b-506b-40d9-b353-a0a38741536f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:23.803402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] The token cache is being invalidate because project cff11022a88846d8a80c1f10dd2b2810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:23.803695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:23.804544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5d57549b-506b-40d9-b353-a0a38741536f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:23.806326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1019/2066] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:57:23 2022] DELETE /identity/v3/projects/cff11022a88846d8a80c1f10dd2b2810 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:23.811373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:23.812050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:23.812050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:23.831512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:23.832661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:23.851268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:23.896131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1048/2067] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:23.909198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:23.910066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:23.910267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:23.935034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:23.937159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:23.948172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1020/2068] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:23.953795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:23.963841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265eed7f-9e79-4d52-95d5-034863826be1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:23.968746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:23.969398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:23.969689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:23.970049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:23.970838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:23.971155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:23.972770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:23.984444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1049/2069] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:57:23.988970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:23.998395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810ddcc-87a5-4a28-91a5-26a39e804ce0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.003051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.003696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.003979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.004252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.004945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.006866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.116378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1021/2070] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:57:23 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:24.120846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.130993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29f0e-9520-4711-9272-b3595590e4d9 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.189998 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.190704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.191009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.191263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.191960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.193858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.306570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1050/2071] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:57:24 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:24.307835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.318366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7926e6-06b0-4ff3-8d5d-6d60dc092187 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.323626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.324298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.324583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.324860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.325623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.327512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.338311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1022/2072] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:24.342400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.352512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a92f75-62c6-490a-af21-ad7190de039c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.357192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.357820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.358124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.358405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.359169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.360763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.371846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1051/2073] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:24.375807 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.386432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78b991-13b4-4711-882b-1c7fb9159a42 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.391117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.391987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.392273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.392555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] PATH_INFO: `/v3/projects/b962f46984a24ff2b98763704aa21abb/users/821615bfa33b4125bf37b3db8be697ee/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.407203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b962f46984a24ff2b98763704aa21abb, user_id=821615bfa33b4125bf37b3db8be697ee, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.409987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03aef4-3c10-4d67-8ba2-b79e4e33799f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.469012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.473343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:24.484837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981dcb-b436-4dcc-ae2f-05dd4b2d601a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.495328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1023/2074] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:57:24 2022] PUT /identity/v3/projects/b962f46984a24ff2b98763704aa21abb/users/821615bfa33b4125bf37b3db8be697ee/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:24.507349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.507748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.508400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.545463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] MFA Rules not processed for user `821615bfa33b4125bf37b3db8be697ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:24.547132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:24.560923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:24.583817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.584696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.585118 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.585256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] PATH_INFO: `/v3/users/1317239afd014696b93d14838fda287b` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.586473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1317239afd014696b93d14838fda287b)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.588584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.644465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1024/2075] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:24.656139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:24.657550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:24.716104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778f59d-fb16-4b02-aa67-88801c09453f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:24.719858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:24.720505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.720794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.721076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.730611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:24.732145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0468a-fee6-4847-b995-7ea0bb07edbc service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:24.742431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1025/2076] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:24.810326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:24.824342 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1317239afd014696b93d14838fda287b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:24.831341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1317239afd014696b93d14838fda287b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:24.832103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1317239afd014696b93d14838fda287b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:24.844051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1317239afd014696b93d14838fda287b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:24.937148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1317239afd014696b93d14838fda287b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:24.944428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1052/2077] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:57:24 2022] DELETE /identity/v3/users/1317239afd014696b93d14838fda287b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:24.949082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:24.949549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:24.949639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:24.967463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:24.968667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.038545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1026/2078] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:25.048815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:25.050502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.109409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78213810-b50c-4ab9-af20-fcb68bbfee28 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:25.113222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:25.113867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.114165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.114440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:25.123673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:25.125130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea8cb5-c927-4fdd-bc6e-aa8b4067c72b service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:25.134493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1053/2079] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:25.701670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:25.703416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.720878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7502ae-bc06-4f5d-ad9d-326a2589d42c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:25.727531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:25.728334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.728533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.728987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] PATH_INFO: `/v3/projects/ffc7e400990f44a4af1367e01ef0cd1e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:25.730580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc7e400990f44a4af1367e01ef0cd1e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:25.732604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:25.870565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:25.872160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc7e400990f44a4af1367e01ef0cd1e', 'request_id': 'req-cdb421bd-c37d-4ea0-bb96-74a0fa520175'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.963639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc7e400990f44a4af1367e01ef0cd1e', 'request_id': 'req-cdb421bd-c37d-4ea0-bb96-74a0fa520175'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.970425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc7e400990f44a4af1367e01ef0cd1e', 'request_id': 'req-cdb421bd-c37d-4ea0-bb96-74a0fa520175'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.983015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] The token cache is being invalidate because project ffc7e400990f44a4af1367e01ef0cd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:25.983313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.984035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb421bd-c37d-4ea0-bb96-74a0fa520175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.985595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1027/2080] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:57:25 2022] DELETE /identity/v3/projects/ffc7e400990f44a4af1367e01ef0cd1e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:25.990433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.990909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.991040 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.010513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:26.013334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.079847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1054/2081] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:26.095893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.099419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.099419 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.125003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:26.126271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.139593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1028/2082] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:26.143818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.154146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59f769-ae1e-437f-8099-642ff05c4d4c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.158783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.159434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.159718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.159996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.160720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:26.161063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.162807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.174425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1055/2083] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:26.178946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.188466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18eb4ef-7640-4399-9dc4-d7c10706fdf5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.194122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.196967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.196967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.196967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.196967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.198680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.287558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1029/2084] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:26.292474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.302578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d460-a1be-4b89-8d2d-e91990bf3b88 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.336509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:26.359859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.360522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.360804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.361079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.361771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.364163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.478841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1056/2085] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:26.482904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.493134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e4dc3-87dc-43e5-a7f6-5cfe0e5da70c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.499573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.500323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.500612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.500920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.501825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.503870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.519407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1030/2086] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:26.520741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.530564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d272de-5c04-4bd8-ad5f-5534fb0d57c2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.535250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.535907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.536193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.536472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.537203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.538882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124f9ae-0935-4d3b-aa99-79170ebb0d92 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.553864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1057/2087] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:26.557273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.568250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49078c9-703b-4b9b-8062-addce58fa4a2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.573729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.574614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.574940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.575218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] PATH_INFO: `/v3/projects/4a00e1e1ecce42468244dcc747ed211e/users/b6822f47e986425da2eb1ba41916e6eb/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.590566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a00e1e1ecce42468244dcc747ed211e, user_id=b6822f47e986425da2eb1ba41916e6eb, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.593260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.665217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1031/2088] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:57:26 2022] PUT /identity/v3/projects/4a00e1e1ecce42468244dcc747ed211e/users/b6822f47e986425da2eb1ba41916e6eb/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:26.675068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04c96-00de-4ec2-aaaa-3d0fd4017fd6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.675392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04c96-00de-4ec2-aaaa-3d0fd4017fd6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.675639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04c96-00de-4ec2-aaaa-3d0fd4017fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.704829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f7e04c96-00de-4ec2-aaaa-3d0fd4017fd6 None None] MFA Rules not processed for user `b6822f47e986425da2eb1ba41916e6eb`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:26.707616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f7e04c96-00de-4ec2-aaaa-3d0fd4017fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.775885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1058/2089] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:26.787219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.788983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.849276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f2bd-b343-4a62-ad43-aa33f5170c40 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.853591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.854235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.854516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.854824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.863566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.864991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.880856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1032/2090] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:35.065573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:35.066074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:35.066345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:35.137017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:35.138239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:35.221709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1059/2091] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:35.282391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:35.285290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:35.303114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:35.308799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:35.390718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003e002-6459-4d05-b116-5b6217c20453 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:35.396982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:35.397857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:35.398162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:35.398606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] PATH_INFO: `/v3/users/b6822f47e986425da2eb1ba41916e6eb` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:35.400646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6822f47e986425da2eb1ba41916e6eb)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:35.402613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:35.424548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:35.436032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f321b-855c-4d3b-8b5c-eaaad8b6d6ed service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:35.443501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:35.444185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:35.444441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:35.444718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:35.456639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:35.458274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8f523-96ea-412c-a488-24d310752e54 service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:35.467182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1033/2092] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:57:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:57:35.670972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6822f47e986425da2eb1ba41916e6eb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:35.678806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6822f47e986425da2eb1ba41916e6eb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:35.679642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6822f47e986425da2eb1ba41916e6eb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:35.694061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6822f47e986425da2eb1ba41916e6eb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:35.765734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6822f47e986425da2eb1ba41916e6eb'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:35.777006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1060/2093] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:57:35 2022] DELETE /identity/v3/users/b6822f47e986425da2eb1ba41916e6eb => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:35.779265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:35.779406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:35.779704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:35.807636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:35.809055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:35.903304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1034/2094] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:35.912034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:35.915547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:36.064432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaec4d-5902-4afe-9b20-27fcd3e5e372 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:36.064432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:36.064432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:36.064432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:36.064829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:36.075710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:36.078334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc81ef-273e-4aca-a136-54a2152a8aca service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:36.096129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1061/2095] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:36.645411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:36.647809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:36.677076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d7970-3297-4e37-b789-db6461cb9874 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:36.682305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:36.683521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:36.685484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:36.685484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] PATH_INFO: `/v3/projects/4a00e1e1ecce42468244dcc747ed211e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:36.685977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a00e1e1ecce42468244dcc747ed211e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:36.690603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:36.835370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a00e1e1ecce42468244dcc747ed211e', 'request_id': 'req-f1eda246-3397-4349-9a20-a065df26d427'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:36.925888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a00e1e1ecce42468244dcc747ed211e', 'request_id': 'req-f1eda246-3397-4349-9a20-a065df26d427'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:36.933957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a00e1e1ecce42468244dcc747ed211e', 'request_id': 'req-f1eda246-3397-4349-9a20-a065df26d427'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:36.946869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] The token cache is being invalidate because project 4a00e1e1ecce42468244dcc747ed211e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:36.947392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:36.948395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:36.950205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1035/2096] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:57:36 2022] DELETE /identity/v3/projects/4a00e1e1ecce42468244dcc747ed211e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:40.920322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc910a-4771-4771-a7b1-e1b44c4290bc None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:40.920853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc910a-4771-4771-a7b1-e1b44c4290bc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:40.921095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc910a-4771-4771-a7b1-e1b44c4290bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:40.939894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0abc910a-4771-4771-a7b1-e1b44c4290bc None None] MFA Rules not processed for user `becb6feb52b643279066aef1a7a9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:40.941264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0abc910a-4771-4771-a7b1-e1b44c4290bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:40.964213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:41.006681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1062/2097] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:43.346831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:43.348247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:43.419928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:43.456885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eda246-3397-4349-9a20-a065df26d427 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:43.467145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:43.467809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:43.468102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:43.468388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:43.470238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:43.495202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:43.499072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:43.520244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1036/2098] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:57:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:51.072252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=14 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:53.629020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=10 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:58.285788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:58.286417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:58.286417 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:58.306531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:58.308146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.363822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1063/2099] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:58.376168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:58.377587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.445063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754b1af-e6c1-4255-8db2-cec8bf3006a1 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:58.450563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:58.451238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:58.451527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:58.451816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:58.459898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:58.462027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:58.470080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1037/2100] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:57:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:58:05.826015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:05.827997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.895214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d637-5854-4a0d-9bb7-0c9e1c52c2aa None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:05.899784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:05.900645 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.900926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.901201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] PATH_INFO: `/v3/users/7792f354950d41a3b0b81267cd2714cc` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.902402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=7792f354950d41a3b0b81267cd2714cc)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:05.904379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.132613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7792f354950d41a3b0b81267cd2714cc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.140333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7792f354950d41a3b0b81267cd2714cc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.141015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7792f354950d41a3b0b81267cd2714cc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.153323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7792f354950d41a3b0b81267cd2714cc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.215957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7792f354950d41a3b0b81267cd2714cc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.224399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1064/2101] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:05 2022] DELETE /identity/v3/users/7792f354950d41a3b0b81267cd2714cc => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:06.513431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:06.524424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be677-0f55-4ede-bec5-eb77ab61b51c service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:06.582543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:06.584741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.584741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.584741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] PATH_INFO: `/v3/projects/f3fe34cb175740b299804b8c9d2f3371` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.585248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3fe34cb175740b299804b8c9d2f3371)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:06.587273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.707418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3fe34cb175740b299804b8c9d2f3371', 'request_id': 'req-ce0babc9-b6cb-42fa-84a0-36b66a44a417'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.878860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3fe34cb175740b299804b8c9d2f3371', 'request_id': 'req-ce0babc9-b6cb-42fa-84a0-36b66a44a417'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.886320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3fe34cb175740b299804b8c9d2f3371', 'request_id': 'req-ce0babc9-b6cb-42fa-84a0-36b66a44a417'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.897422 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] The token cache is being invalidate because project f3fe34cb175740b299804b8c9d2f3371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:06.897773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.898421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ce0babc9-b6cb-42fa-84a0-36b66a44a417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.900056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1038/2102] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:06 2022] DELETE /identity/v3/projects/f3fe34cb175740b299804b8c9d2f3371 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:16.219299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:16.894227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:28.312864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:28.314335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:28.327883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b38d7-26bb-4950-bb4a-bd1a17c1a9da admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:28.385022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:28.385890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:28.386173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:28.386450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] PATH_INFO: `/v3/users/236860f76ea443008dc35bc4a5214d48` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:28.387666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236860f76ea443008dc35bc4a5214d48)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:28.390079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:28.576028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236860f76ea443008dc35bc4a5214d48'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:28.582651 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236860f76ea443008dc35bc4a5214d48'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:28.583317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236860f76ea443008dc35bc4a5214d48'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:28.595217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236860f76ea443008dc35bc4a5214d48'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:28.669803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236860f76ea443008dc35bc4a5214d48'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:28.675971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1065/2103] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:28 2022] DELETE /identity/v3/users/236860f76ea443008dc35bc4a5214d48 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:28.680473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:28.680960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:28.681073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:28.699191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:28.701015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:28.759584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1039/2104] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:28.769650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:28.771082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:28.828376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020458b0-b40e-4551-87f9-7b4907b2ce59 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:28.831890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:28.832528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:28.832810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:28.833087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:28.842401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:28.847601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:28.853453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1066/2105] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:29.282860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:29.286037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:29.298568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006acde9-47d7-48ba-b216-4d35c5f1863c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:29.303501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:29.304356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:29.304640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:29.304913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] PATH_INFO: `/v3/projects/4b4de3aedd894cb7adac85188597fec3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:29.306096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4de3aedd894cb7adac85188597fec3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:29.308604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:29.471044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4de3aedd894cb7adac85188597fec3', 'request_id': 'req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.538562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4de3aedd894cb7adac85188597fec3', 'request_id': 'req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.549240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4de3aedd894cb7adac85188597fec3', 'request_id': 'req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.563155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] The token cache is being invalidate because project 4b4de3aedd894cb7adac85188597fec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:29.563510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.564206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.565941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1040/2106] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:29 2022] DELETE /identity/v3/projects/4b4de3aedd894cb7adac85188597fec3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:29.569915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:29.579879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378773b2-3f2e-46dc-a0c0-d006ea5f70f1 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:29.599351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:29.636621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:29.637483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:29.637763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:29.638033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] PATH_INFO: `/v3/users/0c6851fc9ca54c5abcd40685219a8e8e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:29.639239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6851fc9ca54c5abcd40685219a8e8e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:29.641154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:29.880817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6851fc9ca54c5abcd40685219a8e8e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.888505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6851fc9ca54c5abcd40685219a8e8e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.889126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6851fc9ca54c5abcd40685219a8e8e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.900888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6851fc9ca54c5abcd40685219a8e8e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.969255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-af49ffa3-f7d6-4eb5-aeb7-8b6c4b82e281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6851fc9ca54c5abcd40685219a8e8e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:29.975891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1067/2107] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:29 2022] DELETE /identity/v3/users/0c6851fc9ca54c5abcd40685219a8e8e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:30.374377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:30.376343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:30.389966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b041e7e-1c3e-473e-a9d2-fedd9fb7aaea admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:30.446173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:30.447063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:30.447346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:30.447622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] PATH_INFO: `/v3/projects/8e24e9eebf39444f976ac07679bfbc6c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:30.448792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e24e9eebf39444f976ac07679bfbc6c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:30.451416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:30.528463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:30.543805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e24e9eebf39444f976ac07679bfbc6c', 'request_id': 'req-6b2abcff-6d8b-49bc-a232-65ea74089bdb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:30.612589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e24e9eebf39444f976ac07679bfbc6c', 'request_id': 'req-6b2abcff-6d8b-49bc-a232-65ea74089bdb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:30.620394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e24e9eebf39444f976ac07679bfbc6c', 'request_id': 'req-6b2abcff-6d8b-49bc-a232-65ea74089bdb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:30.633465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] The token cache is being invalidate because project 8e24e9eebf39444f976ac07679bfbc6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:30.633778 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:30.634929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6b2abcff-6d8b-49bc-a232-65ea74089bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:30.636694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1041/2108] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:30 2022] DELETE /identity/v3/projects/8e24e9eebf39444f976ac07679bfbc6c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:30.647201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897057d-80fc-419d-9271-2841e5420980 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:30.647201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897057d-80fc-419d-9271-2841e5420980 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:30.647201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897057d-80fc-419d-9271-2841e5420980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:30.660955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4897057d-80fc-419d-9271-2841e5420980 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:30.662090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4897057d-80fc-419d-9271-2841e5420980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:30.721136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1068/2109] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:30.728772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:30.729060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:30.729338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:30.748974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:30.750178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:30.758083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1042/2110] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:30.762182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897057d-80fc-419d-9271-2841e5420980 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:30.771774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897057d-80fc-419d-9271-2841e5420980 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:30.776387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:30.777034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:30.777328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:30.777615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:30.778354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:30.778700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:30.780343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:30.791705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1069/2111] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:30.796349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:30.805515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb893-9247-4e62-be34-d4e6acf7d309 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:30.810749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:30.811382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:30.811664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:30.811939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:30.812871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:30.814791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:30.951950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1043/2112] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:30 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:30.956060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:30.966457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3ebe-6243-4e33-a426-18fc09a90234 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:30.992056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:31.022289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.022973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.023256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.023530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.024202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.026113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.121366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1070/2113] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:30 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:31.125696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.136103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c7726-3ef3-42d2-bded-cd340091dc0a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.140857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.141539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.141815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.142087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.142910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.144566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.158513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1044/2114] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:31.162627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.173050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfe40c-a0f9-4592-afdf-57bd1ff663bb admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.177596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.178241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.178524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.178827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.179543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.181150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215161c-4731-4cf1-bd21-d7feaa2305a2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.190954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1071/2115] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:31.195169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.210590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f590898-f996-4e66-88ef-b5918e565ec8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.210590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.211372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.211658 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.211937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] PATH_INFO: `/v3/projects/056006ed32fc4782995233ceed6a76ed/users/1ce35e6a1e7d41a1a677803e0d691fcc/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.226870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056006ed32fc4782995233ceed6a76ed, user_id=1ce35e6a1e7d41a1a677803e0d691fcc, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.229526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.298919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1045/2116] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:31 2022] PUT /identity/v3/projects/056006ed32fc4782995233ceed6a76ed/users/1ce35e6a1e7d41a1a677803e0d691fcc/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:31.310075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.310368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.310677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.337539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] MFA Rules not processed for user `1ce35e6a1e7d41a1a677803e0d691fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:31.338785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:31.400761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1072/2117] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:31.405839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.467249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e425-5351-4636-87fa-ebe20ef64121 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.471965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.472610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.472893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.473196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.474011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.475918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.551171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1046/2118] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:31 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:31.555015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.566385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1334-12da-42ad-9ed2-0980c34b1785 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.620921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.621578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.621862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.622691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.623352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.625276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.695538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1073/2119] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:31 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:31.701106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.711091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720fc7-30ce-4da8-8bbb-531fd1e40723 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.715764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.716457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.716745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.717021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.717741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.719358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.733413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1047/2120] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:31.737531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.748302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd5dc3-377d-4980-833f-10c9e0021c2f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.753024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.753675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.753962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.754238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.755005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.756601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115252-631c-4228-aeac-debf7945e0be admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.767430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1074/2121] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:31.771632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:31.781671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedd4b6-30b7-4796-a46e-5b1bc8a8ba59 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:31.786329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:31.787358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.787651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.787930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] PATH_INFO: `/v3/projects/9fb14a3f846b469bbf373c8a074b4957/users/60a962e24f7f4272b4db42022e8f87e3/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.804636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb14a3f846b469bbf373c8a074b4957, user_id=60a962e24f7f4272b4db42022e8f87e3, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:31.807319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:31.862399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:31.867894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1048/2122] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:31 2022] PUT /identity/v3/projects/9fb14a3f846b469bbf373c8a074b4957/users/60a962e24f7f4272b4db42022e8f87e3/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:31.880585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:31.880877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:31.881155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:31.907498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] MFA Rules not processed for user `60a962e24f7f4272b4db42022e8f87e3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:31.908696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:31.912750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:31.978479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1075/2123] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:31.983626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.044918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf0f57-a432-4884-818e-9afe7ee17479 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.049510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.050153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.050457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.050923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.051641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.053542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.177035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1049/2124] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:31 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:32.181143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.192701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d355d-7c31-468a-a50f-07d6032bd46c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.247133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.248218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.248505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.248785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.249466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.251526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.358858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1076/2125] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:32 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:32.362719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.374056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc412bfc-5e74-42cb-9149-d850c0c7b8e6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.378892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.379663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.379945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.380226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.380948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.382534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.393059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1050/2126] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:32.397078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.407623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b83dff-e92f-4692-aa29-ccb648bfe751 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.412144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.413006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.413290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.413563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] PATH_INFO: `/v3/projects/f96bae4019be4399bed49175bfc23a71/users/d03fbd8ad8a843c8b37b2d35c158c572/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.428971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96bae4019be4399bed49175bfc23a71, user_id=d03fbd8ad8a843c8b37b2d35c158c572, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.431698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.506408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1077/2127] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:32 2022] PUT /identity/v3/projects/f96bae4019be4399bed49175bfc23a71/users/d03fbd8ad8a843c8b37b2d35c158c572/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:32.510367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.520517 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c0146-9c21-423d-bb00-c481be98668e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.575067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.575753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.576036 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.576309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.577019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.578593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7591dc0-b2d0-4e0d-9afd-c53d68106e7f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.589977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1051/2128] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:32.594627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.605058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce2bf3-7998-430e-852d-dd8da23dc70e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.610331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.611243 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.611590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.611916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] PATH_INFO: `/v3/projects/f96bae4019be4399bed49175bfc23a71/users/d03fbd8ad8a843c8b37b2d35c158c572/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.613986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96bae4019be4399bed49175bfc23a71, user_id=d03fbd8ad8a843c8b37b2d35c158c572, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.616752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.627788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1078/2129] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:32 2022] PUT /identity/v3/projects/f96bae4019be4399bed49175bfc23a71/users/d03fbd8ad8a843c8b37b2d35c158c572/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:32.642969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.643286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.643569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.671985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] MFA Rules not processed for user `d03fbd8ad8a843c8b37b2d35c158c572`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:32.673201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:32.734244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:32.744273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1052/2130] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:32.755229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:32.757017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:32.814191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828598c-7966-4b29-83dd-74f04c9ba553 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:32.817826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:32.818465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:32.818785 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:32.819069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:32.828562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:32.829994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:32.834851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:32.839735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1079/2131] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:33.858478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:33.859971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:33.871855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b8dac-6a4c-4f59-8a88-a36331b0b770 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:33.876360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:33.877011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:33.877327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:33.877608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:33.935363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:33.936811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:33.946044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1053/2132] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:42.319973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:42.328854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46347ea1-8ce6-4cb7-816c-bdc7f3aed1c6 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:42.333088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:42.333713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:42.333994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:42.334270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:42.389898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:42.391374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e6dd-86be-41c5-8de3-19d120362a58 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:42.400771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1080/2133] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:42.766518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:42.768012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:42.836416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861c54-8fe1-4e44-a708-13edf48e8521 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:42.840974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:42.841839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:42.842119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:42.842393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] PATH_INFO: `/v3/users/821615bfa33b4125bf37b3db8be697ee` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:42.843645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] RBAC: Authorizing `identity:delete_user(user_id=821615bfa33b4125bf37b3db8be697ee)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:42.845580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:43.060483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821615bfa33b4125bf37b3db8be697ee'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:43.068178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821615bfa33b4125bf37b3db8be697ee'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:43.068867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821615bfa33b4125bf37b3db8be697ee'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:43.080947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821615bfa33b4125bf37b3db8be697ee'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:43.201363 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821615bfa33b4125bf37b3db8be697ee'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:43.207967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1054/2134] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:42 2022] DELETE /identity/v3/users/821615bfa33b4125bf37b3db8be697ee => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:43.212466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:43.212767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:43.213054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:43.230167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:43.233987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:43.290068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1081/2135] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:43.300685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:43.302263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:43.318899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:43.365428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752ecd-63da-4dd5-b8dd-e6441eea373f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:43.369104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:43.369735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:43.370025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:43.370302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:43.380255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:43.381723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6320e0-6f59-48b6-9a18-22d55cbe777b service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:43.391104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1055/2136] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:43.787684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:43.789105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:43.802559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe02900-2633-4681-8309-12aa16db9515 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:43.807878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:43.808736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:43.809018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:43.809320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] PATH_INFO: `/v3/projects/b962f46984a24ff2b98763704aa21abb` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:43.810606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b962f46984a24ff2b98763704aa21abb)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:43.813115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:43.910825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b962f46984a24ff2b98763704aa21abb', 'request_id': 'req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.009141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b962f46984a24ff2b98763704aa21abb', 'request_id': 'req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.015409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b962f46984a24ff2b98763704aa21abb', 'request_id': 'req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.026547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] The token cache is being invalidate because project b962f46984a24ff2b98763704aa21abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:44.026930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.027579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c5e3df3d-1f7b-4fa4-bb14-36f3f1292b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.029115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1082/2137] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:43 2022] DELETE /identity/v3/projects/b962f46984a24ff2b98763704aa21abb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:44.034127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.034611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.034756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.052662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:44.055598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.117543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1056/2138] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:44.125030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.125316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.125594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.145413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:44.146595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.154362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1083/2139] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:44.158505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.168834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c7ac-82cf-4496-99d4-7173038de741 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.173348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.174005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.174306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.174567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.175955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:44.176268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.177866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.189593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1057/2140] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:44.194421 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.203511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822005c6-195e-4e96-a14e-b8acfd84028c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.208756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.209394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.209682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.209955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.210628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.212606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.225353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:44.313159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1084/2141] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:44.317904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.328082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865f49e-3e85-4ea5-9763-152bf377001d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.385006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.385679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.385970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.386271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.387041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.389018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.475176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1058/2142] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:44.479571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.489684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a494722-4faf-4c85-b480-8eff9912a031 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.494268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.494943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.495229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.495507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.496233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.497846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.509046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1085/2143] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:44.512796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.523298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d975bf6-c172-42e8-8478-c81df6471ddf admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.527855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.528501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.528789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.529066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.529791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.531415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e05c8b-92ff-42ff-9395-778b59480b0b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.541229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1059/2144] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:44.545451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.556808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba600faf-cc23-453e-96af-4254e848c0a4 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.561530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.562672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.562955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.563230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] PATH_INFO: `/v3/projects/bb0ee163dc0c4237aaa4e2efb85878fc/users/479781403bb94b9788fd589f838fea3f/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.578148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0ee163dc0c4237aaa4e2efb85878fc, user_id=479781403bb94b9788fd589f838fea3f, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.581697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.687550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1086/2145] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:44 2022] PUT /identity/v3/projects/bb0ee163dc0c4237aaa4e2efb85878fc/users/479781403bb94b9788fd589f838fea3f/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:44.693127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.696439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.696439 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.711290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:44.712479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.729267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:44.770864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1060/2146] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:44.780383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.782837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.839184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f358-ed18-4b0d-bd26-557544c71877 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.842975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.843614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.843898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.844176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.853447 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.854966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da6494-b0df-4ede-ba4e-642e79b94bb5 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.864714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1087/2147] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:46.420741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.422230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:46.435420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6528d-556f-4854-b69a-a415dbd200d9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.440257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.441195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.441480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.441788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] PATH_INFO: `/v3/users/1ce35e6a1e7d41a1a677803e0d691fcc` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.443012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce35e6a1e7d41a1a677803e0d691fcc)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.448295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.621153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce35e6a1e7d41a1a677803e0d691fcc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:46.628524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce35e6a1e7d41a1a677803e0d691fcc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:46.629241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce35e6a1e7d41a1a677803e0d691fcc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:46.641456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce35e6a1e7d41a1a677803e0d691fcc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:46.717613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce35e6a1e7d41a1a677803e0d691fcc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:46.724989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1061/2148] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:46 2022] DELETE /identity/v3/users/1ce35e6a1e7d41a1a677803e0d691fcc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:46.729790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.730268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.730368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.748475 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:46.749782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:46.824075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1088/2149] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:46.832156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.836305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:46.908255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f05-19c3-4e13-8a3c-e946c9fa66d5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.912993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.913647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.913934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.914216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.924466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.925994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.937941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1062/2150] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:47.442458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:47.442458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:47.454668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adc05c-fbfc-41da-a083-fdd22a92e40e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:47.460045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:47.461045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:47.461331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:47.461638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] PATH_INFO: `/v3/projects/056006ed32fc4782995233ceed6a76ed` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:47.462849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056006ed32fc4782995233ceed6a76ed)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:47.465314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:47.619507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056006ed32fc4782995233ceed6a76ed', 'request_id': 'req-48bf9b63-7ca3-4030-86a8-9b7266f06070'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.678212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056006ed32fc4782995233ceed6a76ed', 'request_id': 'req-48bf9b63-7ca3-4030-86a8-9b7266f06070'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.680176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:47.686091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056006ed32fc4782995233ceed6a76ed', 'request_id': 'req-48bf9b63-7ca3-4030-86a8-9b7266f06070'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.699858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] The token cache is being invalidate because project 056006ed32fc4782995233ceed6a76ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:47.700212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.700906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.702603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1089/2151] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:47 2022] DELETE /identity/v3/projects/056006ed32fc4782995233ceed6a76ed => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:47.706847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:47.717147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54004b29-71d2-4d57-adc7-0f164f1fb612 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:47.761487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:47.791628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:47.792548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:47.792830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:47.793108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] PATH_INFO: `/v3/users/60a962e24f7f4272b4db42022e8f87e3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:47.794347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a962e24f7f4272b4db42022e8f87e3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:47.796667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:47.977161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a962e24f7f4272b4db42022e8f87e3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.986672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a962e24f7f4272b4db42022e8f87e3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.988685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a962e24f7f4272b4db42022e8f87e3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.005536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a962e24f7f4272b4db42022e8f87e3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.072339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a962e24f7f4272b4db42022e8f87e3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.080180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1063/2152] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:47 2022] DELETE /identity/v3/users/60a962e24f7f4272b4db42022e8f87e3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:48.524006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.526723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.552056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf9b63-7ca3-4030-86a8-9b7266f06070 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.628376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.629322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.629659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.629942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] PATH_INFO: `/v3/projects/9fb14a3f846b469bbf373c8a074b4957` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.631210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb14a3f846b469bbf373c8a074b4957)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.633160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:48.731724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb14a3f846b469bbf373c8a074b4957', 'request_id': 'req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.809714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb14a3f846b469bbf373c8a074b4957', 'request_id': 'req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.818587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb14a3f846b469bbf373c8a074b4957', 'request_id': 'req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.837781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] The token cache is being invalidate because project 9fb14a3f846b469bbf373c8a074b4957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:48.837981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.838802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.840694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1090/2153] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:48 2022] DELETE /identity/v3/projects/9fb14a3f846b469bbf373c8a074b4957 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:48.845841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.847800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.860618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0eb1c-0c26-4aaf-94f8-55e518ad1677 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.930411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.931662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.932025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.932443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] PATH_INFO: `/v3/users/d03fbd8ad8a843c8b37b2d35c158c572` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.934684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03fbd8ad8a843c8b37b2d35c158c572)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.936185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:49.150071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03fbd8ad8a843c8b37b2d35c158c572'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.161974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03fbd8ad8a843c8b37b2d35c158c572'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.171189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03fbd8ad8a843c8b37b2d35c158c572'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.177610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:49.218504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03fbd8ad8a843c8b37b2d35c158c572'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.358915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ee737e14-5181-42c8-b2df-395ec170beb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03fbd8ad8a843c8b37b2d35c158c572'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.371949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1064/2154] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:58:48 2022] DELETE /identity/v3/users/d03fbd8ad8a843c8b37b2d35c158c572 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:50.032136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.032136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.056186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00d76c-16a2-4bc4-bbf4-13c4348d1e1c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.134316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.135225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.135518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.135854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] PATH_INFO: `/v3/projects/f96bae4019be4399bed49175bfc23a71` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.138758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96bae4019be4399bed49175bfc23a71)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.142790 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.167309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:50.279081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96bae4019be4399bed49175bfc23a71', 'request_id': 'req-8ce28185-cfa0-43af-8c51-f38fc5860883'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:50.410063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96bae4019be4399bed49175bfc23a71', 'request_id': 'req-8ce28185-cfa0-43af-8c51-f38fc5860883'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:50.421444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96bae4019be4399bed49175bfc23a71', 'request_id': 'req-8ce28185-cfa0-43af-8c51-f38fc5860883'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:50.440121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] The token cache is being invalidate because project f96bae4019be4399bed49175bfc23a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:50.440665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:50.443089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ce28185-cfa0-43af-8c51-f38fc5860883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:50.450343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1091/2155] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:58:50 2022] DELETE /identity/v3/projects/f96bae4019be4399bed49175bfc23a71 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:50.451260 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a25cf-9b3f-4453-994e-bb2f9458f0e6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.451716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a25cf-9b3f-4453-994e-bb2f9458f0e6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.454228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a25cf-9b3f-4453-994e-bb2f9458f0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.488566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-478a25cf-9b3f-4453-994e-bb2f9458f0e6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:50.493040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-478a25cf-9b3f-4453-994e-bb2f9458f0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.609692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1065/2156] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:50.618305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.618681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.618898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.636288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.636571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.636857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.646718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:50.648271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.658683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1092/2157] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:50.663036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.676856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d9e8a-e8bc-4bcb-8165-6f2bd9110446 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.685445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] MFA Rules not processed for user `479781403bb94b9788fd589f838fea3f`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:50.685445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.685668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.685668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.685668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.685668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.686019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:50.686019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.686775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.700793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1093/2158] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:50.704817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.717987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4eb89-0673-4194-bb0c-2c7c99d17fc8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.724273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.724950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.725235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.725512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.727663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.729633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.770207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1066/2159] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:50.774783 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.776171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.789939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7fe12-ec6c-4bfb-b96e-f456d662ef40 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.799109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.799109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.799109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.799109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.799767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.799767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.826680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1094/2160] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:50.830255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.913532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1067/2161] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:50.914897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.917346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fee7-02ce-4f31-b405-96abd10673b1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.009644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81e85e-6f7b-48f5-83d9-2134eb363df0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.015100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.015100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.015100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.016019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.016720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.017213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.017764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.017882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.018488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.026961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.027097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.027097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.036727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:51.135162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1068/2162] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:51.135358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1095/2163] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:51.135776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.137891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.146743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c213a6-68da-45ad-8bff-93d726d57001 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.150436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a427fbb-0b43-464b-babd-dd5753d500ce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.151581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.152250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.152670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.152950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.153687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.155328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.158307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.158307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.158307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.158307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.158826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.159912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.166108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1069/2164] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.171798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.172114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1096/2165] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.176940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.190792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813bc54-64c5-4446-a85c-10c633465012 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.192289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28e2c9-03f6-48f7-9d36-8010aabaa8a8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.193706 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.194370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.194714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.194982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.195681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.197395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.197797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.198801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.199004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.199281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] PATH_INFO: `/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6/users/a9767369aefb4b1684c6aad00190f675/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.209028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1097/2166] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:51.213215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.215713 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4acf7c7eb4248df881d3f014ec44bf6, user_id=a9767369aefb4b1684c6aad00190f675, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.219466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.225066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8dfb0-067a-47bb-8dc8-a8685012a349 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.230415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.231612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.231899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.232184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] PATH_INFO: `/v3/projects/fa4be84bccd3414bb3248046bb72d55a/users/f7b33ef9f51c42c3b012817fe4f00e2a/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.247977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4be84bccd3414bb3248046bb72d55a, user_id=f7b33ef9f51c42c3b012817fe4f00e2a, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.250741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bfd94-b23d-4467-91e5-e60769e6b962 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.296218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1070/2167] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6/users/a9767369aefb4b1684c6aad00190f675/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:51.299828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.369509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1098/2168] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/fa4be84bccd3414bb3248046bb72d55a/users/f7b33ef9f51c42c3b012817fe4f00e2a/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:51.374125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd775d80-2744-4d75-b0bf-a056018dce43 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.381125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.381454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.381889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.408406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] MFA Rules not processed for user `f7b33ef9f51c42c3b012817fe4f00e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:51.410127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.435454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.436634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.437288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.437927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.438827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.441756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.459614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1071/2169] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.463934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.478959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524ff83-c1e8-4d8d-a571-fb92f2675d03 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.484914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.485880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.490046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.490046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] PATH_INFO: `/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6/users/a9767369aefb4b1684c6aad00190f675/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.490046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4acf7c7eb4248df881d3f014ec44bf6, user_id=a9767369aefb4b1684c6aad00190f675, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.490772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1099/2170] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:51.491452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.494204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.506166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1072/2171] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6/users/a9767369aefb4b1684c6aad00190f675/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:51.515739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bba99-27fe-4a4c-86c5-bb1d73db12cd None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.578089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.578800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.579088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.579370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.580070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.582872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.697564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1100/2172] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:51.701940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.712843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e31f3-442e-44bd-a224-8f65e89b1ec5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.772488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.773164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.773495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.773785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.774483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.776441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.873165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1073/2173] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:51.882256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.889715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727547bb-013b-4b2e-81c6-71544caa0a98 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.895677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.896460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.896901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.897236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.898081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.900230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.912300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1101/2174] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:51.916196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.926460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff790224-8e12-4f7d-b929-8999e6b6a0df admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.931238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.932100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.932387 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.932671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] PATH_INFO: `/v3/projects/c9035fd558504fa5a782b531c786a212/users/055ae850f36446bf9364212d58b10072/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.948314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9035fd558504fa5a782b531c786a212, user_id=055ae850f36446bf9364212d58b10072, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.951066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.032626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1074/2175] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/c9035fd558504fa5a782b531c786a212/users/055ae850f36446bf9364212d58b10072/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:52.036010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.052191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e3d3c-3bc1-40f1-84bc-300d802792f8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.129173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.129859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.130149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.130432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.131205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.132821 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e0936-fda5-4806-b3c3-df50946f913d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.144500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1102/2176] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:52.148649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.160989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615de0ee-4ca5-4fb2-981a-8a64755b8e96 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.166117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.167054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.167426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.167759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] PATH_INFO: `/v3/projects/c9035fd558504fa5a782b531c786a212/users/055ae850f36446bf9364212d58b10072/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.168539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:52.170312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9035fd558504fa5a782b531c786a212, user_id=055ae850f36446bf9364212d58b10072, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.173098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.191990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1075/2177] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:58:52 2022] PUT /identity/v3/projects/c9035fd558504fa5a782b531c786a212/users/055ae850f36446bf9364212d58b10072/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:52.205607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864475-4b4c-4541-a02b-39486ec053c4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.205923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864475-4b4c-4541-a02b-39486ec053c4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.211275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864475-4b4c-4541-a02b-39486ec053c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.247999 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6e864475-4b4c-4541-a02b-39486ec053c4 None None] MFA Rules not processed for user `055ae850f36446bf9364212d58b10072`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:52.249252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6e864475-4b4c-4541-a02b-39486ec053c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.335178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1103/2178] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:52.344610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.347972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.414194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16556091-a525-4fae-9c80-694298fbb0ec admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.418059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.418697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.418986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.419266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.480676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.482585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.493303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1076/2179] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:52.761433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:54.219678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:58.285363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df0369-5fce-4d48-93e4-5687d9e3ec65 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.285984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df0369-5fce-4d48-93e4-5687d9e3ec65 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.285984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df0369-5fce-4d48-93e4-5687d9e3ec65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.313296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f8df0369-5fce-4d48-93e4-5687d9e3ec65 None None] MFA Rules not processed for user `a9767369aefb4b1684c6aad00190f675`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:58.314509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f8df0369-5fce-4d48-93e4-5687d9e3ec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.392127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1104/2180] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:58.404680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:58.406194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.447699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:58.487043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4a76-578f-4b05-a93f-92e050059c39 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.490786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.491443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.491724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.492006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.548088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.549515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.558774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1077/2181] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:58.936765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.940184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.940184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.956779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:58.958020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.978148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:59.018546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1105/2182] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:59.228136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:59.230207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:59.318115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44b36e-42cf-4be2-8630-8d55b1c4e8ab service glance] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:59.332834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:59.333425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:59.333973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:59.334164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:59.345699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:59.348054 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7854b-e2e4-4068-b54e-1e937b0264d0 service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:59.358093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1078/2183] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:58:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:59:01.497782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.500211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.518149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d66f53-b2fb-4963-a357-bf25fa276a2d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.522750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.523866 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.524555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.525080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.538083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.545772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.569499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1106/2184] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:06.507475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bc728-8842-4860-a284-c207abb5223f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:06.508080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bc728-8842-4860-a284-c207abb5223f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:06.508080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bc728-8842-4860-a284-c207abb5223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:06.526531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-5f6bc728-8842-4860-a284-c207abb5223f None None] MFA Rules not processed for user `cd25c0ad8e1c4fa08eca9b6bddeca55d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:06.527909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5f6bc728-8842-4860-a284-c207abb5223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:06.595522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1079/2185] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:09.041594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.148173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31243de8-ffec-4e38-a8d5-87ab7a7018d7 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.150606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.151548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.151834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.152114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] PATH_INFO: `/v3/users/f7b33ef9f51c42c3b012817fe4f00e2a` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.153576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b33ef9f51c42c3b012817fe4f00e2a)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.155626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.634372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b33ef9f51c42c3b012817fe4f00e2a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:09.642698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b33ef9f51c42c3b012817fe4f00e2a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:09.643698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b33ef9f51c42c3b012817fe4f00e2a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:09.668577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b33ef9f51c42c3b012817fe4f00e2a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:09.780250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b33ef9f51c42c3b012817fe4f00e2a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:09.791696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1107/2186] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:59:09 2022] DELETE /identity/v3/users/f7b33ef9f51c42c3b012817fe4f00e2a => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:09.793073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.793549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.793830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.802308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:09.814627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:09.816270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:09.893459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1080/2187] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:09.899871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.901453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:09.949752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:09.975197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5013e-cc5a-44b3-b5c4-48b9c688c5a0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.979382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.980136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.980493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.980882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.993020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.000220 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.020158 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1108/2188] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:10.574232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.575672 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:10.593835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85282e9b-69e3-45df-8fae-7b88126275d9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.596742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.597733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.598024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.598303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] PATH_INFO: `/v3/projects/fa4be84bccd3414bb3248046bb72d55a` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.599725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4be84bccd3414bb3248046bb72d55a)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.601838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.757143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4be84bccd3414bb3248046bb72d55a', 'request_id': 'req-4ec982e1-0376-4477-b9e6-6765e48997d5'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.842980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4be84bccd3414bb3248046bb72d55a', 'request_id': 'req-4ec982e1-0376-4477-b9e6-6765e48997d5'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.851459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4be84bccd3414bb3248046bb72d55a', 'request_id': 'req-4ec982e1-0376-4477-b9e6-6765e48997d5'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.880552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] The token cache is being invalidate because project fa4be84bccd3414bb3248046bb72d55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:10.880949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.881677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4ec982e1-0376-4477-b9e6-6765e48997d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.883366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1081/2189] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:59:10 2022] DELETE /identity/v3/projects/fa4be84bccd3414bb3248046bb72d55a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:10.887375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.900986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557afe6-ccfd-4d2f-9e77-0be3f9a9662c service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.955800 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.956675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.956962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.957244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] PATH_INFO: `/v3/users/055ae850f36446bf9364212d58b10072` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.958438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=055ae850f36446bf9364212d58b10072)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.960433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:11.010432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:11.010795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:11.011075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:11.029608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] MFA Rules not processed for user `479781403bb94b9788fd589f838fea3f`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:11.033006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.090229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1082/2190] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:11.168845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055ae850f36446bf9364212d58b10072'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:11.177128 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055ae850f36446bf9364212d58b10072'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:11.177951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055ae850f36446bf9364212d58b10072'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:11.190400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055ae850f36446bf9364212d58b10072'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:11.286839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3d491889-35a5-4204-aae1-8800b03b4f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055ae850f36446bf9364212d58b10072'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:11.293907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1109/2191] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:59:10 2022] DELETE /identity/v3/users/055ae850f36446bf9364212d58b10072 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:11.805602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:11.807140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.826176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a7f89-05c9-4ef1-bf15-8bd568c91757 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:11.877765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:11.944408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:11.945409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:11.946055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:11.946395 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] PATH_INFO: `/v3/projects/c9035fd558504fa5a782b531c786a212` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:11.948745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9035fd558504fa5a782b531c786a212)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:11.951941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.089555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9035fd558504fa5a782b531c786a212', 'request_id': 'req-d3006f8b-c145-403e-b392-e906492c8b7e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.183739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9035fd558504fa5a782b531c786a212', 'request_id': 'req-d3006f8b-c145-403e-b392-e906492c8b7e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.187543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9035fd558504fa5a782b531c786a212', 'request_id': 'req-d3006f8b-c145-403e-b392-e906492c8b7e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.198681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] The token cache is being invalidate because project c9035fd558504fa5a782b531c786a212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:12.198976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.199622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3006f8b-c145-403e-b392-e906492c8b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.201175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1083/2192] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:59:11 2022] DELETE /identity/v3/projects/c9035fd558504fa5a782b531c786a212 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:12.207860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.208359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.208460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.226439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:12.227815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:12.281936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:12.291217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1110/2193] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:12.301256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.301834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.302406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.325893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:12.327187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:12.337719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1084/2194] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:12.347025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.352218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671617e-8e02-4113-adef-7dbb3c57493c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.359013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.359756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.360062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.360320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.361119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:12.361430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.365223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.382356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1111/2195] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:12.384804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.394216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c3353-029a-45c1-83f1-2ba021fd2db8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.399063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.399714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.400003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.400283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.400984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.402944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.505622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1085/2196] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:59:12 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:12.506803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.517940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef303cbc-3503-422c-b59c-38beddcf6edc admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.596731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.596731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.596731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.596731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.597154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.597154 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.745175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1112/2197] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:59:12 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:59:12.749475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.762727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1838-25d7-4790-a9bd-0a45d6b29482 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.768386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.769318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.769608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.769885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.770615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.772253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.786274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1086/2198] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:12.790083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.801539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d3b08-22ca-4237-b16c-302bf38f5689 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.806330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.807164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.807346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.807624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.808357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.810005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668458-5472-43f4-980c-f2d540b28e0e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.821075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1113/2199] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:12.825018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:12.836925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a4ddc-0b1e-462b-8308-28a64831963b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.841522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.842409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.842797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.843080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] PATH_INFO: `/v3/projects/b823917994e94d509d9425571ddaee45/users/5de77d1a390c402e95b412a7b11e8fa0/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.858874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b823917994e94d509d9425571ddaee45, user_id=5de77d1a390c402e95b412a7b11e8fa0, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.861549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.970264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1087/2200] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:59:12 2022] PUT /identity/v3/projects/b823917994e94d509d9425571ddaee45/users/5de77d1a390c402e95b412a7b11e8fa0/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:12.983133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1e8b-08c8-4fed-97ac-cc147036dacc None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.983552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1e8b-08c8-4fed-97ac-cc147036dacc None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.983796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1e8b-08c8-4fed-97ac-cc147036dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:13.016470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-108f1e8b-08c8-4fed-97ac-cc147036dacc None None] MFA Rules not processed for user `5de77d1a390c402e95b412a7b11e8fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:13.017702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-108f1e8b-08c8-4fed-97ac-cc147036dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:13.094542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1114/2201] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:13.106962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:13.108875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:13.171664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1de96-6b1b-45f6-865a-4f1faac17b2e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:13.175513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:13.176195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:13.176485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:13.176765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:13.185913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:13.190665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:13.197776 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1088/2202] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:14.821994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25b015-b5d5-45b2-890e-c3a53fcafd5b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:14.823344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25b015-b5d5-45b2-890e-c3a53fcafd5b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:14.824103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25b015-b5d5-45b2-890e-c3a53fcafd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:14.853579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7a25b015-b5d5-45b2-890e-c3a53fcafd5b None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:14.855042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7a25b015-b5d5-45b2-890e-c3a53fcafd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:14.915479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:14.925648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1115/2203] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:14.940137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:14.944205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:14.952590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:15.076742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3575b-edef-431f-9f57-951178fa1343 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:15.083006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666278-024f-4474-8d56-def61bef839d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:15.083740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666278-024f-4474-8d56-def61bef839d service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:15.083998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666278-024f-4474-8d56-def61bef839d service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:15.084278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666278-024f-4474-8d56-def61bef839d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:15.094692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6666278-024f-4474-8d56-def61bef839d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:15.096898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6666278-024f-4474-8d56-def61bef839d service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:15.110758 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1089/2204] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:59:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:59:15.751777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:15.752072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:15.752911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:15.773850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:15.775255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:15.788558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1116/2205] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:15.894554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666278-024f-4474-8d56-def61bef839d service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:15.910104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666278-024f-4474-8d56-def61bef839d service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:15.916929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:15.918270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:15.918479 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:15.918936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:15.929490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:15.931474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:15.943098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1090/2206] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:59:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:59:25.766471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:25.938208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=37 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:34.308810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:34.320485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:34.411188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf105f6-62fa-4c34-8ced-e4bf282f11c5 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:34.415731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:34.416401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:34.416693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:34.416978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:34.418994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:34.538853 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:34.542269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:34.566084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1117/2207] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:37.190491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:37.200557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a8efb-cf6f-423f-a1c4-06fffd70a48e service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:37.205176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:37.206493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:37.207404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:37.207867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:37.214629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:37.237139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:37.238573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:37.252077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1091/2208] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:44.562298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=63 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:45.126079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:45.128177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:45.153183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356960-e03f-48da-9486-711381f573b0 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:45.168909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:45.168909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:45.168909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:45.168909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:45.169236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:45.227437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:45.229137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931c142-e2b0-4603-a903-5f734b144709 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:45.239918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1118/2209] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:47.247549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:48.624642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:48.692526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf75f-6f79-441a-b810-d24d16903d52 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:48.697109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:48.697976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:48.698262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:48.698569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] PATH_INFO: `/v3/users/5de77d1a390c402e95b412a7b11e8fa0` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:48.699786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de77d1a390c402e95b412a7b11e8fa0)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:48.701716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:48.879663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de77d1a390c402e95b412a7b11e8fa0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:48.886808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de77d1a390c402e95b412a7b11e8fa0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:48.887511 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de77d1a390c402e95b412a7b11e8fa0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:48.903307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de77d1a390c402e95b412a7b11e8fa0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:49.030381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de77d1a390c402e95b412a7b11e8fa0'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:49.038740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1092/2210] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:59:48 2022] DELETE /identity/v3/users/5de77d1a390c402e95b412a7b11e8fa0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:49.043827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:49.044295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:49.044740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:49.063721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:49.065036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:49.148199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1119/2211] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:49.158962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:49.160605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:49.222482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f17-fe75-49cf-a6ac-0e80e4bd7f51 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:49.226146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:49.226835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:49.227122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:49.227403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:49.236715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:49.239060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4542f-58f4-4add-9606-f1f522489ad1 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:49.249383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1093/2212] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:49.724590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:49.726924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:49.755492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25101102-254e-4d90-a02b-619c8c150d9e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:49.761619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:49.762783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:49.763223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:49.763669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] PATH_INFO: `/v3/projects/b823917994e94d509d9425571ddaee45` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:49.765671 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b823917994e94d509d9425571ddaee45)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:49.768340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:49.915728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b823917994e94d509d9425571ddaee45', 'request_id': 'req-523fa716-d7df-4f06-9426-7c4b27555e16'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:50.028972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b823917994e94d509d9425571ddaee45', 'request_id': 'req-523fa716-d7df-4f06-9426-7c4b27555e16'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:50.041252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b823917994e94d509d9425571ddaee45', 'request_id': 'req-523fa716-d7df-4f06-9426-7c4b27555e16'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:50.056643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] The token cache is being invalidate because project b823917994e94d509d9425571ddaee45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:50.058054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:50.059198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:50.061700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1120/2213] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:59:49 2022] DELETE /identity/v3/projects/b823917994e94d509d9425571ddaee45 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:50.067056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6b62-8181-4dc8-9a46-c4b4fc3d41c1 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.067562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6b62-8181-4dc8-9a46-c4b4fc3d41c1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.067697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6b62-8181-4dc8-9a46-c4b4fc3d41c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.092445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-404b6b62-8181-4dc8-9a46-c4b4fc3d41c1 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:50.093653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-404b6b62-8181-4dc8-9a46-c4b4fc3d41c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:50.104995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.108051 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:50.115081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:50.176071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:50.187545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1094/2214] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:50.197278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.197960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.198459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.229697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:50.231244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:50.243346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1095/2215] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:50.247943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.258375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24075b3c-6eb9-4e94-b68f-59849265d7f2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.260071 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa716-d7df-4f06-9426-7c4b27555e16 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.266047 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.267065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.267426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.267802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.268196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.269567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:50.270085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.270992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.271434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.272470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.272826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.275348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:50.288072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1096/2216] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:50.293175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.308433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b91447-f175-4c69-9c01-5ee9e8dafa76 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.313802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.314510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.314829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.315113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.315885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.317912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.362598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.364241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.378668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1121/2217] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:50.452320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1097/2218] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:59:50 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:50.457406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.471782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d8ee-3d8a-4e80-a860-b859ed566463 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.564311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.565659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.566059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.566455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.567319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.569808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.663269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1122/2219] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:59:50 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:50.666833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.679448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea7b15-c030-4875-ad5a-5f82856c2b0a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.686472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.687169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.687453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.687729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.688453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.690662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.704570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1098/2220] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:50.706137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.716980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522cef-ed1d-414f-8945-f24d1ca607df admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.722520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.723720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.724326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.724608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.725346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.727236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9092ec-4915-4408-ad16-6e88ecea3d67 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.739193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1123/2221] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:50.747289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:50.759852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc40f69-5365-4a99-b933-ee260d747d11 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:50.765689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:50.766580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.767102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.767311 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] PATH_INFO: `/v3/projects/490ef63ce58349e48c879995f99c1c20/users/fafa7b89f5a447f6a053cf89cb21e486/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.785305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490ef63ce58349e48c879995f99c1c20, user_id=fafa7b89f5a447f6a053cf89cb21e486, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:50.788102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:50.945411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1099/2222] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:59:50 2022] PUT /identity/v3/projects/490ef63ce58349e48c879995f99c1c20/users/fafa7b89f5a447f6a053cf89cb21e486/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:59:50.955683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:50.955915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:50.956226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:50.982319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] MFA Rules not processed for user `fafa7b89f5a447f6a053cf89cb21e486`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:50.983575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:51.022244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:51.071925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1124/2223] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:51.079627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:51.081440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:51.150838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54215256-78c2-47c4-8e7e-1cd54ed19c01 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:51.154536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:51.155209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:51.155497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:51.155780 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:51.164850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:51.166318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368e06d-5d2e-45a2-a374-1ba693ec38cf service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:51.175642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1100/2224] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:51.206817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:51.269762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695147a-8da4-43ad-ae04-b27e61206eee None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:51.274357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:51.275252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:51.275539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:51.275847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] PATH_INFO: `/v3/users/fafa7b89f5a447f6a053cf89cb21e486` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:51.277053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafa7b89f5a447f6a053cf89cb21e486)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:51.279029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:51.501323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafa7b89f5a447f6a053cf89cb21e486'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:51.508516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafa7b89f5a447f6a053cf89cb21e486'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:51.509214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafa7b89f5a447f6a053cf89cb21e486'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:51.524628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafa7b89f5a447f6a053cf89cb21e486'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:51.630093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafa7b89f5a447f6a053cf89cb21e486'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:51.636702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1125/2225] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 14:59:51 2022] DELETE /identity/v3/users/fafa7b89f5a447f6a053cf89cb21e486 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:51.641269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:51.641566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:51.641851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:51.660212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:51.661427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:51.712462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:51.728610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1101/2226] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:51.739258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:51.741782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:51.799284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:51.805424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6301-2b01-4184-8167-5b869942e174 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:51.809063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:51.809714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:51.810124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:51.810411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:51.820810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:51.822259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:51.832625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1126/2227] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:52.051262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:52.053249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.072861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a0ce-51e7-44bc-88bb-c98df71317b8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:52.073697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:52.074443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.074762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.075172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.078350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.090854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:52.092384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d160-61a3-44de-b195-c988993bc7a9 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:52.101135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1102/2228] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:52.363958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:52.365390 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.380820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f67e0-d4f8-40e2-919f-3b71f7cb51ea service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:52.385812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:52.386812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.387096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.387359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] PATH_INFO: `/v3/projects/490ef63ce58349e48c879995f99c1c20` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.388553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=490ef63ce58349e48c879995f99c1c20)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:52.390620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:52.537325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490ef63ce58349e48c879995f99c1c20', 'request_id': 'req-16740172-1e1a-46d7-8a04-19176648e8cd'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:52.573828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf479-817c-43a2-a906-e05c2e1e5e84 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.574394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf479-817c-43a2-a906-e05c2e1e5e84 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.574757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcf479-817c-43a2-a906-e05c2e1e5e84 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.577796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1103/2229] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:59:52.591928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.592631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.595038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.648421 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490ef63ce58349e48c879995f99c1c20', 'request_id': 'req-16740172-1e1a-46d7-8a04-19176648e8cd'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:52.665197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:52.667007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490ef63ce58349e48c879995f99c1c20', 'request_id': 'req-16740172-1e1a-46d7-8a04-19176648e8cd'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:52.667153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.686627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] The token cache is being invalidate because project 490ef63ce58349e48c879995f99c1c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:52.687530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:52.688694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-16740172-1e1a-46d7-8a04-19176648e8cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:52.691060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1127/2230] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 14:59:52 2022] DELETE /identity/v3/projects/490ef63ce58349e48c879995f99c1c20 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:52.704077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54290e2b-06db-4dd4-b66f-d369793cab11 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.709969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54290e2b-06db-4dd4-b66f-d369793cab11 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.709969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54290e2b-06db-4dd4-b66f-d369793cab11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.732663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-54290e2b-06db-4dd4-b66f-d369793cab11 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:52.734663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-54290e2b-06db-4dd4-b66f-d369793cab11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.759892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1104/2231] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:52.774899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:52.776381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.803072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1128/2232] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:52.811569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.811859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.812142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.832110 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:52.833352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:52.836446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf2109-a5cb-40aa-a34a-79c4a2d78ece None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:52.840213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:52.840870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.841139 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.841417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.841864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1129/2233] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:52.846310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:52.853205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:52.854676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:52.855439 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611f17b-b6d8-489b-889d-e773ffeb97ad None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:52.860074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:52.860704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.860987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.861293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.862010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:52.862320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:52.863958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:52.866091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1105/2234] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:52.875776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1130/2235] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 14:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:59:52.880131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:52.891199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf7be-999b-4396-a6e3-7ebd23f8042d service glance] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:52.896794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:52.897455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:52.897734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:52.898014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:52.898741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:52.900902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381ffa-08fb-4948-9165-2ac20e5be191 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.023475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1106/2236] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 14:59:52 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:59:53.024666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.034909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52ffd-34f4-4933-82aa-95113466d920 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.089084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:53.098061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.098993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.099425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.099855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.100711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.102839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.105252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb50c-b99d-4258-aaef-11008ffd0259 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.105728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb50c-b99d-4258-aaef-11008ffd0259 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.105913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb50c-b99d-4258-aaef-11008ffd0259 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.107405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1107/2237] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:59:53.112830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.113122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.117127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.134190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:53.135543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.170121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:53.197625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1108/2238] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:53.205565 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1131/2239] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 14:59:53 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:53.209655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.214844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.217305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.229565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb51d-10fa-4400-9d1e-87989ffd3b28 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.232196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699047f-3f82-4716-9c0d-3e90191d958e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.234095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.234787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.235068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.235350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.235815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.237082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.237372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.237649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.238702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.240527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.243344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.244780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.255309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1109/2240] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:53.255488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1132/2241] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:53.256474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.266391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9934b-c92b-4baf-b918-d0942b571d17 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.271033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.271685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.271975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.272246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.272955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.274594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19012c90-16cd-45ee-992e-059633781b28 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.285190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1110/2242] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:53.289306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.301991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217380c-fedd-49c3-9881-9e0fe2a09027 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.307554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.308482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.308735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.309013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] PATH_INFO: `/v3/projects/99c1c69b004d499fa0c4af9fe5d7959f/users/17b8801e4f9b43758e0cea8ea3ef30f9/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.332092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c1c69b004d499fa0c4af9fe5d7959f, user_id=17b8801e4f9b43758e0cea8ea3ef30f9, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.335003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d66e7f-991e-4462-81eb-f5eb2a4e5328 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.351303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b264eb-eb6c-4a01-ad0c-aa17d775ec7d None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.351596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b264eb-eb6c-4a01-ad0c-aa17d775ec7d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.351880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b264eb-eb6c-4a01-ad0c-aa17d775ec7d None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.353299 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1111/2243] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:59:53.358954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.359508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.359795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.386095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:53.387541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.403348 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1133/2244] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 14:59:53 2022] PUT /identity/v3/projects/99c1c69b004d499fa0c4af9fe5d7959f/users/17b8801e4f9b43758e0cea8ea3ef30f9/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:53.414245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.414544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.415203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.445341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] MFA Rules not processed for user `17b8801e4f9b43758e0cea8ea3ef30f9`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:53.446552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.465986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1112/2245] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:53.477743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.487545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f3040-6643-46d4-b549-aa1ba28ce221 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.514255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1134/2246] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:53.525751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.527324 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.538146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.538333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.538613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.538915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.587496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a316625-f5ca-4dc0-bc4a-af67411300ff None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.591372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.592023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.592312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.592614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.601623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.603126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.606374 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:53.608851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.610286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.612389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1135/2247] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:53.621461 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1113/2248] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:53.687282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.688750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.733878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:53.765195 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d26d-3be4-4181-beaa-8b5a243bb9b9 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.771495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.772560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.772855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.773138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.776629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.793434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.793780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.805701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1136/2249] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:54.239846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:54.241433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:54.260577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bf959-7fd6-434c-92a8-55871d15d5d7 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:54.270207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:54.271226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:54.271724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:54.272308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:54.275060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:54.344929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:54.347822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0840a1-0b38-4666-9086-0b932329cd81 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:54.358314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1114/2250] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:58.372633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:58.374170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:58.386033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f0615-da6b-4a06-a2a9-811245929139 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:58.389739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:58.390379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:58.390685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:58.390967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:58.393231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:58.403429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:58.404865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:58.413627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1137/2251] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:59.328794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:59.329293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:59.329397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:59.347136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:59.348320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:59.405130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1115/2252] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:59.427992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:59.429387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:59.502883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd46b9-5f62-4bf0-a413-0eaf8da63c7d service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:59.507907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:59.508552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:59.508835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:59.509109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:59.517053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:59.518684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:59.527044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1138/2253] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 14:59:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:00:08.512259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:08.514350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:08.533202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f4927-2c92-4e6f-9213-80e70f3c6d10 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:08.539868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:08.540589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:08.540874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:08.541226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:08.543133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:08.608969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:08.610557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79058e5c-bf3c-4579-ab91-5fd041fcdfec service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:08.619636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1116/2254] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:09.504183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:09.571307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4485d-6040-4966-8cf3-971e01ecb0c0 service swift] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:09.576044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:09.576921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:09.577204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:09.577478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] PATH_INFO: `/v3/users/17b8801e4f9b43758e0cea8ea3ef30f9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:09.578702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b8801e4f9b43758e0cea8ea3ef30f9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:09.582746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:09.755960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:09.757662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b8801e4f9b43758e0cea8ea3ef30f9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:09.764551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b8801e4f9b43758e0cea8ea3ef30f9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:09.765250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b8801e4f9b43758e0cea8ea3ef30f9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:09.777601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b8801e4f9b43758e0cea8ea3ef30f9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:09.835748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b8801e4f9b43758e0cea8ea3ef30f9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:09.842516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1139/2255] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:00:09 2022] DELETE /identity/v3/users/17b8801e4f9b43758e0cea8ea3ef30f9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:00:09.847198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:09.847712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:09.847847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:09.868728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:00:09.870067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:09.909719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:09.938493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1117/2256] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:00:09.949503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:09.951341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:10.010401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daca9f-0ad0-4fd5-9df6-46ed55c91e0c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:10.014070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:10.014827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:10.015123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:10.015402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:10.024841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:10.026307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:10.036591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1140/2257] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:10.518076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:10.520130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:10.533351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eb3ee-5cf8-4efc-acd9-fc7d4fa1b2b5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:10.539162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:10.540183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:10.540475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:10.540771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] PATH_INFO: `/v3/projects/99c1c69b004d499fa0c4af9fe5d7959f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:10.547925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c1c69b004d499fa0c4af9fe5d7959f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:10.550964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:10.677364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c1c69b004d499fa0c4af9fe5d7959f', 'request_id': 'req-d2fb589a-e216-4d02-a908-6a935c9a9fb6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:10.764885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c1c69b004d499fa0c4af9fe5d7959f', 'request_id': 'req-d2fb589a-e216-4d02-a908-6a935c9a9fb6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:10.773188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c1c69b004d499fa0c4af9fe5d7959f', 'request_id': 'req-d2fb589a-e216-4d02-a908-6a935c9a9fb6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:10.787039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] The token cache is being invalidate because project 99c1c69b004d499fa0c4af9fe5d7959f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:00:10.787425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:10.788158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:00:10.790084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1118/2258] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:00:10 2022] DELETE /identity/v3/projects/99c1c69b004d499fa0c4af9fe5d7959f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:00:20.031764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:20.782609 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:27.135101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:27.136538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:27.148143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b4cb-9790-4906-9d3b-3f4ff70d5fe7 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:27.197721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:27.198347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:27.198687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:27.198975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:27.200974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:27.264172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:27.265878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cec30d-d7e7-41a2-90bd-d6b46d43ac6f service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:27.276737 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1141/2259] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:00:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:29.557859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:29.560385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:29.637966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb589a-e216-4d02-a908-6a935c9a9fb6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:29.644691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:29.645358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:29.645697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:29.646172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:29.648152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:29.662147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:29.663800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:29.675640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1119/2260] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:00:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:30.245730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:30.246224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:30.246525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:30.275907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:00:30.277164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:30.361730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1142/2261] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:00:30.376338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:30.378427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:30.473487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c888b4-a324-4cd4-9286-45bb0a5a8a45 service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:30.479464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:30.480378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:30.480732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:30.481084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:30.491704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:30.493357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a321b86-c2ac-45cf-97e7-1fac7442211d service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:30.521415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1120/2262] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:00:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:00:33.156296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:33.186305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce05ca-ce58-4129-8c75-44b389a9556c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:33.203322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:33.204990 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:33.205508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:33.206223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:33.208454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:33.237247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:33.239707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00c5a5-6a56-498c-b9da-f9c53a81354d service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:33.257735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1143/2263] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:00:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:40.501314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=66 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:43.248630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=81 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:55.983927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:55.984522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:55.984522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:56.002606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] MFA Rules not processed for user `becb6feb52b643279066aef1a7a9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:00:56.004073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:56.066371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1121/2264] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:01:06.058248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=25 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:53.190124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978881db-f1cb-428b-81c8-20daedf4a611 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:53.190601 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978881db-f1cb-428b-81c8-20daedf4a611 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:53.190719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978881db-f1cb-428b-81c8-20daedf4a611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:53.209315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-978881db-f1cb-428b-81c8-20daedf4a611 None None] MFA Rules not processed for user `237510469a8e4e36ad40a34cdf705656`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:01:53.210865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-978881db-f1cb-428b-81c8-20daedf4a611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:53.268917 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1144/2265] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:01:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:01:53.591843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:53.593860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:53.607789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48048a-ec87-4783-bed1-718a06ab9fd1 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:53.611472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:53.612265 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:53.612634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:53.612916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:53.614760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:53.624427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:53.625884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:53.636356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1122/2266] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:01:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:02.622131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:02.622607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:02.622735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:02.640318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] MFA Rules not processed for user `cd25c0ad8e1c4fa08eca9b6bddeca55d`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:02.641522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:02.701922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1145/2267] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:03.632670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:05.226149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:05.246257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc0c87-4bfb-44af-b4fd-ffe697865a99 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:05.246257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:05.246257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:05.246257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:05.246717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:05.248320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:05.260715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:05.262385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:05.273195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1123/2268] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:12.690276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=50 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:15.268768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:34.579611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:34.581032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:34.592915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ca5bb-bacc-4255-85d5-fbf3eec6dc54 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:34.597487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:34.597487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:34.597487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:34.597816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:34.599431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:34.662770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:34.664278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:34.674506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1146/2269] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:02:38.493069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:38.494556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:38.573527 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3aa7-30c0-4e2c-b4e5-e6a5218dbd43 service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:38.578851 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:38.579504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:38.579791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:38.580073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:38.581889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:38.593762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:38.602675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:38.606365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1124/2270] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:38.677527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:38.677527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:38.677527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1dae1-a9fa-4b13-be8e-a859e784aff7 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:38.678275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:38.678275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:38.678275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:38.678275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] PATH_INFO: `/v3/users/becb6feb52b643279066aef1a7a9f063` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:38.678661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=becb6feb52b643279066aef1a7a9f063)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:38.678661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:39.035332 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becb6feb52b643279066aef1a7a9f063'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.042860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becb6feb52b643279066aef1a7a9f063'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.043729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becb6feb52b643279066aef1a7a9f063'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.056665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becb6feb52b643279066aef1a7a9f063'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.166250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-88ca3a60-82b5-4f00-a8ff-e918875d2f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becb6feb52b643279066aef1a7a9f063'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.173233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1147/2271] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:02:38 2022] DELETE /identity/v3/users/becb6feb52b643279066aef1a7a9f063 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:39.499997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:39.511037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51174bbf-ad23-45ec-b448-220202c7a0f4 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:39.573227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:39.574179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:39.574519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:39.574876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] PATH_INFO: `/v3/projects/9263e6d657dd458d9f497bb868adb941` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:39.581024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9263e6d657dd458d9f497bb868adb941)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:39.581024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:39.695250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9263e6d657dd458d9f497bb868adb941', 'request_id': 'req-3de2e810-816f-4040-a4e6-33b358a66391'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.772078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9263e6d657dd458d9f497bb868adb941', 'request_id': 'req-3de2e810-816f-4040-a4e6-33b358a66391'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.780124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9263e6d657dd458d9f497bb868adb941', 'request_id': 'req-3de2e810-816f-4040-a4e6-33b358a66391'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.796569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] The token cache is being invalidate because project 9263e6d657dd458d9f497bb868adb941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:39.796814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.797525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-3de2e810-816f-4040-a4e6-33b358a66391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:39.799196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1125/2272] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:02:39 2022] DELETE /identity/v3/projects/9263e6d657dd458d9f497bb868adb941 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:39.804102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119e81d-d494-473d-b338-36feba61a02d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:39.804920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119e81d-d494-473d-b338-36feba61a02d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:39.805384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119e81d-d494-473d-b338-36feba61a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:39.824217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b119e81d-d494-473d-b338-36feba61a02d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:39.826272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b119e81d-d494-473d-b338-36feba61a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:39.910580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1148/2273] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:39.913109 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:39.917287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:39.917790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:39.939737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:39.942858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:39.959650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1126/2274] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:39.963809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119e81d-d494-473d-b338-36feba61a02d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:39.974824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119e81d-d494-473d-b338-36feba61a02d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:39.980812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:39.983360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:39.983360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:39.983360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:39.983360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:02:39.983974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:39.985694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:39.987865 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:40.003615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1149/2275] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:02:40.005410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.016644 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f1b86-ffc4-43f1-882c-59a1eafae470 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.021794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.022603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.023018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.023433 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.024259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.026301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.044164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:40.180184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1127/2276] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:40 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:02:40.184817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.194784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d418b1-777a-4699-90aa-344456efc87d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.266362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.267229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.267519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.267806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.268519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.270492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.358265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1150/2277] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:40 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:40.362613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.373626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f647-9d53-40e0-94d8-1c0e9c4f535d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.379251 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.379928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.380216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.380497 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.381273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.383100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.397148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1128/2278] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:02:40.398382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.408843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36d0e-62e6-4ae6-bd66-abc760a9f094 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.413856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.414544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.414868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.415354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.416090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.417738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e055d-a089-4105-90a7-98e9d4bb63fb admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.429114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1151/2279] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:40.433111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.444165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a142b-cd50-4c1a-8ef2-afdf9502f970 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.448920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.449739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.450019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.451229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] PATH_INFO: `/v3/projects/03785b08439e46f7a11528031401b5c7/users/8def0c04143d4cd7bd606d846e28eef3/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.466909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=03785b08439e46f7a11528031401b5c7, user_id=8def0c04143d4cd7bd606d846e28eef3, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.469597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.561947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1129/2280] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:40 2022] PUT /identity/v3/projects/03785b08439e46f7a11528031401b5c7/users/8def0c04143d4cd7bd606d846e28eef3/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:40.577141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0779614-3627-46d4-8d46-860104854335 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.577442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0779614-3627-46d4-8d46-860104854335 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.577725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0779614-3627-46d4-8d46-860104854335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.612654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d0779614-3627-46d4-8d46-860104854335 None None] MFA Rules not processed for user `8def0c04143d4cd7bd606d846e28eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:40.613906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d0779614-3627-46d4-8d46-860104854335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:40.690700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1152/2281] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:40.695607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.763153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5c74-541d-4995-89e5-b0bceeaae3bd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.768074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.768666 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.772285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.772285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.772285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.772285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.895698 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1130/2282] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:40 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:40.896857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0779614-3627-46d4-8d46-860104854335 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:40.908020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0779614-3627-46d4-8d46-860104854335 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:40.968857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:40.969573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:40.969856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:40.970146 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:40.970884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:40.972902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:40.982830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:41.083667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1153/2283] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:40 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:41.087153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:41.098177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff37880-e187-4fc0-ab47-e331fedd0fe3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:41.102849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:41.103539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.103803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.104090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.104881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:41.107059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:41.118129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1131/2284] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:41.122245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:41.132277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950a100-d654-49a3-b794-9e8d78fd0c4d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:41.136993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:41.138148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.138288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.138502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] PATH_INFO: `/v3/projects/23969df391984f878c30dfcc0fbfec7d/users/7855f08ac4014158a949bf4b642c98fa/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.154735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=23969df391984f878c30dfcc0fbfec7d, user_id=7855f08ac4014158a949bf4b642c98fa, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:41.157449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:41.246770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1154/2285] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:41 2022] PUT /identity/v3/projects/23969df391984f878c30dfcc0fbfec7d/users/7855f08ac4014158a949bf4b642c98fa/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:41.250715 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:41.260824 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba05d64-e55d-4edf-8597-b48198372f4d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:41.317380 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:41.318058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.318345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.318656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.319382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:41.321000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c321120-9ede-4e2a-8c07-8034d4b88994 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:41.331525 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1132/2286] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:41.335593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:41.346459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86971767-e929-4650-b53c-e041a2ee78ce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:41.351074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:41.352317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.352603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.352881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] PATH_INFO: `/v3/projects/23969df391984f878c30dfcc0fbfec7d/users/7855f08ac4014158a949bf4b642c98fa/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.354898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23969df391984f878c30dfcc0fbfec7d, user_id=7855f08ac4014158a949bf4b642c98fa, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:41.358421 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:41.369290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1155/2287] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:41 2022] PUT /identity/v3/projects/23969df391984f878c30dfcc0fbfec7d/users/7855f08ac4014158a949bf4b642c98fa/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:41.379412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.379708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.379994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.407738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] MFA Rules not processed for user `7855f08ac4014158a949bf4b642c98fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:41.408943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:41.414206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:41.478548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1133/2288] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:41.489749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:41.491639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:41.552575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dce28-5170-4fcf-8194-c753efd0c24d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:41.556034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:41.556718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.557005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.557286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.568607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:41.570372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da969e3-83d7-4d09-9e72-09cd4e6d022c service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:41.582326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1156/2289] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:42.703213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:42.768748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ef47-624d-43b5-81eb-84006d80b392 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:42.773379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:42.774245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:42.774535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:42.774859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] PATH_INFO: `/v3/users/8def0c04143d4cd7bd606d846e28eef3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:42.776096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8def0c04143d4cd7bd606d846e28eef3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:42.778032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:43.027135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8def0c04143d4cd7bd606d846e28eef3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:43.034623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8def0c04143d4cd7bd606d846e28eef3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:43.035304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8def0c04143d4cd7bd606d846e28eef3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:43.048203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8def0c04143d4cd7bd606d846e28eef3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:43.128325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8def0c04143d4cd7bd606d846e28eef3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:43.138078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1134/2290] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:02:42 2022] DELETE /identity/v3/users/8def0c04143d4cd7bd606d846e28eef3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:43.139732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:43.140226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:43.140346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:43.159191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:43.160413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:43.223617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1157/2291] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:43.234478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:43.236002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:43.265226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:43.298142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7c096-c998-49c0-adf6-d106b7d5dfa2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:43.302064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:43.302757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:43.303044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:43.303323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:43.314179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:43.318664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:43.328141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1135/2292] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:43.812452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:43.814271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:43.828649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ceb8-ca6d-4acd-9b6e-7d050a0eff82 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:43.834793 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:43.835777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:43.836144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:43.836524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] PATH_INFO: `/v3/projects/03785b08439e46f7a11528031401b5c7` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:43.837838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=03785b08439e46f7a11528031401b5c7)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:43.840193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:43.996815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03785b08439e46f7a11528031401b5c7', 'request_id': 'req-76c69556-e2a8-41f4-9a21-e872fe4239ce'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.000461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:44.145818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03785b08439e46f7a11528031401b5c7', 'request_id': 'req-76c69556-e2a8-41f4-9a21-e872fe4239ce'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.152370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03785b08439e46f7a11528031401b5c7', 'request_id': 'req-76c69556-e2a8-41f4-9a21-e872fe4239ce'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.165087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] The token cache is being invalidate because project 03785b08439e46f7a11528031401b5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:44.165440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.166239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.168098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1158/2293] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:02:43 2022] DELETE /identity/v3/projects/03785b08439e46f7a11528031401b5c7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:44.172889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:44.183007 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4ff4e-40f9-490d-9757-473d5847e277 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:44.245701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:44.246507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:44.246913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:44.247190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] PATH_INFO: `/v3/users/7855f08ac4014158a949bf4b642c98fa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:44.248462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7855f08ac4014158a949bf4b642c98fa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:44.250437 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:44.542407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7855f08ac4014158a949bf4b642c98fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.550841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7855f08ac4014158a949bf4b642c98fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.551478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7855f08ac4014158a949bf4b642c98fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.565386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7855f08ac4014158a949bf4b642c98fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.670092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2db744f6-38a4-44ae-9dec-c5dfb4a92737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7855f08ac4014158a949bf4b642c98fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:44.677256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1136/2294] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:02:44 2022] DELETE /identity/v3/users/7855f08ac4014158a949bf4b642c98fa => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:45.176246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:45.176246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:45.179283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c69556-e2a8-41f4-9a21-e872fe4239ce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:45.250162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:45.251115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.251758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.251758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] PATH_INFO: `/v3/projects/23969df391984f878c30dfcc0fbfec7d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.253459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=23969df391984f878c30dfcc0fbfec7d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:45.255618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:45.389112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23969df391984f878c30dfcc0fbfec7d', 'request_id': 'req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:45.488826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23969df391984f878c30dfcc0fbfec7d', 'request_id': 'req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:45.495904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23969df391984f878c30dfcc0fbfec7d', 'request_id': 'req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:45.507995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] The token cache is being invalidate because project 23969df391984f878c30dfcc0fbfec7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:45.508286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:45.509008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ad1d0a2a-e0f7-4b62-af85-c8d8f8ef7f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:45.510682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1159/2295] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:02:45 2022] DELETE /identity/v3/projects/23969df391984f878c30dfcc0fbfec7d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:45.517226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.517754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.517919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.536723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:45.538023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:45.589077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:45.601516 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1137/2296] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:45.609723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229279-b73d-4fe4-9b24-28c8f2a847c6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.612107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229279-b73d-4fe4-9b24-28c8f2a847c6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.612107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229279-b73d-4fe4-9b24-28c8f2a847c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.629986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-50229279-b73d-4fe4-9b24-28c8f2a847c6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:45.631501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-50229279-b73d-4fe4-9b24-28c8f2a847c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:45.640625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1160/2297] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:45.645369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:45.655668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0d691-bc67-46f5-9975-48b96b22557d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:45.660687 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:45.661337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.661639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.661918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.662661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:02:45.662996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:45.664576 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:45.674120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.674414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.674736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.676768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1138/2298] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:02:45.682198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:45.693448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c953-d661-4ad8-a3f5-2fa04704d747 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:45.698452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] MFA Rules not processed for user `9dba831ebb584de5a0c84f39901d3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:45.699042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:45.699670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:45.699794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.699979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.700817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.701529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:45.703489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:45.755353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1161/2299] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:02:45.831117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1139/2300] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:45 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:02:45.832310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:45.844537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9ed6-4acf-45ef-9acb-b6c69e385cd1 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:45.849637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:45.903018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:45.903926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:45.904350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:45.904685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:45.905394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:45.907381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:45.990423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1162/2301] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:45 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:45.991987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.004577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7249d7-262b-4f12-a96e-d68f81fad94b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.009907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.010684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.013481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.013481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.013481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.013769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.025102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1140/2302] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:46.029171 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.039443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd0db-6a0a-421f-ae5a-6f862ed3f947 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.044672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.045365 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.045669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.045937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.046699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.048322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6c0db-2214-4763-9dbc-f9b72440a884 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.059578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1163/2303] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:46.065105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.075324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9bf57-9f2f-4b9c-8b4b-91994587aa25 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.080175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.081064 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.081353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.081927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] PATH_INFO: `/v3/projects/e15a7d26257d408db7cd0e6eebf5a965/users/f71dce8182fc445492668141912a507f/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.098386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15a7d26257d408db7cd0e6eebf5a965, user_id=f71dce8182fc445492668141912a507f, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.101250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.232667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1141/2304] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:46 2022] PUT /identity/v3/projects/e15a7d26257d408db7cd0e6eebf5a965/users/f71dce8182fc445492668141912a507f/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:46.244358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.244889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.244984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.279588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] MFA Rules not processed for user `f71dce8182fc445492668141912a507f`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:46.281137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:46.357086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1164/2305] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:46.366907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.431487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4411a7-13a7-4361-b28b-1c187eadee96 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.436535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.440562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.440562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.440562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.440562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.440562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.541290 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1142/2306] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:46 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:46.542412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.555669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9471e9-54db-4eb4-b4da-d2d231c69599 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.629385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.630508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.630941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.631280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.631985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.633927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.724461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1165/2307] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:46 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:46.728728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.741806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8f18b-54ed-4f09-8cd3-c6d5167f9da3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.748016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.748694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.749128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.749279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.749971 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.751793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.764013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1143/2308] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:46.768533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.779216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528d95-fb13-4cd4-aac0-1242a64e0de3 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.784003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.786017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.786017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.786017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] PATH_INFO: `/v3/projects/243fca38d86246ad87260f7e42ca8550/users/aec561a3665b4cd6a03c8a2885858264/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.787930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:46.802595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243fca38d86246ad87260f7e42ca8550, user_id=aec561a3665b4cd6a03c8a2885858264, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.805320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:46.917238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1166/2309] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:46 2022] PUT /identity/v3/projects/243fca38d86246ad87260f7e42ca8550/users/aec561a3665b4cd6a03c8a2885858264/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:02:46.923635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:46.932152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34706081-dab7-4453-8e89-847069bf0da7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:46.977677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:46.989738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:46.990405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:46.990726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:46.991006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:46.991731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:46.993333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f8933-0c50-4111-bc35-39e4d07e3857 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:47.004125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1144/2310] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:47.008399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:47.020004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec90d32-ad54-4518-9e86-dffb15a8db66 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:47.025751 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:47.026950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:47.027599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:47.028001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] PATH_INFO: `/v3/projects/243fca38d86246ad87260f7e42ca8550/users/aec561a3665b4cd6a03c8a2885858264/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:47.031121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243fca38d86246ad87260f7e42ca8550, user_id=aec561a3665b4cd6a03c8a2885858264, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:47.034786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:47.050279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1167/2311] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:47 2022] PUT /identity/v3/projects/243fca38d86246ad87260f7e42ca8550/users/aec561a3665b4cd6a03c8a2885858264/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:47.060771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:47.061108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:47.061472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:47.088397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] MFA Rules not processed for user `aec561a3665b4cd6a03c8a2885858264`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:47.089614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:47.162061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1145/2312] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:47.175030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:47.177505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:47.237728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e0f0-7bd9-4ae1-a95e-c27be1669887 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:47.241599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:47.242276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:47.242578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:47.242974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:47.252838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:47.254386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:47.264643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1168/2313] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:47.665998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:47.667635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:47.745287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76c6a-bc14-449e-81cc-06b891e76af4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:47.751110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:47.751757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:47.752167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:47.752458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:47.754232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:47.766982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:47.768514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efab2c-4cf4-4c20-b515-e16401ab4f77 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:47.780677 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1146/2314] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:48.433561 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:48.510123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522189ce-9d3a-4875-b8c1-fc51b9ffee4e service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:48.516320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:48.517403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:48.517767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:48.518312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] PATH_INFO: `/v3/users/f71dce8182fc445492668141912a507f` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:48.520113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71dce8182fc445492668141912a507f)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:48.523107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:48.730598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71dce8182fc445492668141912a507f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.740303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71dce8182fc445492668141912a507f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.741063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71dce8182fc445492668141912a507f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.756235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71dce8182fc445492668141912a507f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.759479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:48.848259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71dce8182fc445492668141912a507f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.854782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1169/2315] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:02:48 2022] DELETE /identity/v3/users/f71dce8182fc445492668141912a507f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:48.859960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:48.860453 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:48.860566 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:48.880428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:48.881699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:48.937509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:48.946950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1147/2316] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:48.957357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:48.958882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.027783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7105-c5f8-4a2c-baa3-1c68ed149241 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.032272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.033006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.036323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.036323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.043446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.044967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.055939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1170/2317] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:49.522886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.524411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.537297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4944e3-186a-4f8c-854a-674aa6ca574b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.542318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.543209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.543572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.543801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] PATH_INFO: `/v3/projects/e15a7d26257d408db7cd0e6eebf5a965` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.548924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e15a7d26257d408db7cd0e6eebf5a965)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.548924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.706420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e15a7d26257d408db7cd0e6eebf5a965', 'request_id': 'req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.795413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e15a7d26257d408db7cd0e6eebf5a965', 'request_id': 'req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.802339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e15a7d26257d408db7cd0e6eebf5a965', 'request_id': 'req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.815907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] The token cache is being invalidate because project e15a7d26257d408db7cd0e6eebf5a965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:49.816197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.816868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.821155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1148/2318] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:02:49 2022] DELETE /identity/v3/projects/e15a7d26257d408db7cd0e6eebf5a965 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:49.822824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.837905 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd2e60-b54a-43c2-90dd-50c79ac12cda service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.911996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.913104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.913568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.913923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] PATH_INFO: `/v3/users/aec561a3665b4cd6a03c8a2885858264` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.915301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec561a3665b4cd6a03c8a2885858264)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.917357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:50.263709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec561a3665b4cd6a03c8a2885858264'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:50.275344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec561a3665b4cd6a03c8a2885858264'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:50.276186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec561a3665b4cd6a03c8a2885858264'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:50.289103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec561a3665b4cd6a03c8a2885858264'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:50.454527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0bffed9a-7d4a-4a3c-8a1d-d27507645a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec561a3665b4cd6a03c8a2885858264'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:50.463327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1171/2319] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:02:49 2022] DELETE /identity/v3/users/aec561a3665b4cd6a03c8a2885858264 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:50.996167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:50.999922 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:51.025238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57a5bb-f18c-4c6f-83b6-dcb8f1737a61 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:51.125458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:51.126383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:51.126714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:51.127017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] PATH_INFO: `/v3/projects/243fca38d86246ad87260f7e42ca8550` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:51.128360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243fca38d86246ad87260f7e42ca8550)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:51.130489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:51.412667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243fca38d86246ad87260f7e42ca8550', 'request_id': 'req-38c79ad8-ffa2-4005-a132-ba092a32b5f2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:51.591669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243fca38d86246ad87260f7e42ca8550', 'request_id': 'req-38c79ad8-ffa2-4005-a132-ba092a32b5f2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:51.599604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243fca38d86246ad87260f7e42ca8550', 'request_id': 'req-38c79ad8-ffa2-4005-a132-ba092a32b5f2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:51.611889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] The token cache is being invalidate because project 243fca38d86246ad87260f7e42ca8550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:51.612279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:51.613021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-38c79ad8-ffa2-4005-a132-ba092a32b5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:51.615030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1149/2320] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:02:50 2022] DELETE /identity/v3/projects/243fca38d86246ad87260f7e42ca8550 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:51.620159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:51.620687 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:51.620782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:51.644598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:51.646036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:51.656919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:51.712370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1172/2321] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:51.720862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:51.721153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:51.721437 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:51.740491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:51.741727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:51.749729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1150/2322] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:51.754945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:51.765062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c5b6a-5ecb-47ae-a3ba-87e39b59439d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:51.771351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:51.772099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:51.772457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:51.772904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:51.773715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:02:51.774125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:51.775918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:51.789090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1173/2323] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:02:51.793619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:51.806227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd3d99-da1c-4fb0-af96-f55e07624374 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:51.811198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:51.811895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:51.812180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:51.816348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:51.816348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:51.816348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:51.818507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:51.940788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1151/2324] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:51 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:51.944794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:51.955526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee8482-a540-448f-b1eb-d3bc514c9316 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:52.015664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:52.016358 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.016657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.017141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.017873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:52.019974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:52.184670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1174/2325] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:51 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:52.186166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:52.197869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0edd2-ba1b-4046-bca9-ada3e7887d3b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:52.204810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:52.205613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.206490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.206490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.207211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:52.208977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:52.232384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1152/2326] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:52.235407 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:52.248032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7286e-47bb-4527-a9a6-e810dc607fae admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:52.253534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:52.254281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.254664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.255022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.255840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:52.257538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1823703-b088-4baa-8bd1-00b1da1772ec admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:52.269758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1175/2327] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:52.278913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:52.290834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2fa71-7a22-4fe8-9b6d-4fb1cb3aede5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:52.295907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:52.296897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.297216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.297521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] PATH_INFO: `/v3/projects/1f74ce84ddcc4403b96438e1cc8ce523/users/9c1d5ae78d2247ee8afbd26bcc1a79b3/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.320650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f74ce84ddcc4403b96438e1cc8ce523, user_id=9c1d5ae78d2247ee8afbd26bcc1a79b3, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:52.323424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:52.437530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1153/2328] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:52 2022] PUT /identity/v3/projects/1f74ce84ddcc4403b96438e1cc8ce523/users/9c1d5ae78d2247ee8afbd26bcc1a79b3/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:52.442701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656dbb-6ec7-4cb5-9da4-0e4665b3552c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.443000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656dbb-6ec7-4cb5-9da4-0e4665b3552c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.443290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656dbb-6ec7-4cb5-9da4-0e4665b3552c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.461640 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5e656dbb-6ec7-4cb5-9da4-0e4665b3552c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:52.462892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5e656dbb-6ec7-4cb5-9da4-0e4665b3552c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:52.525249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1176/2329] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:52.535286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:52.537016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:52.596908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006a01-15d3-43ae-ad57-f1f048099e6b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:52.600567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:52.601215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:52.601504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:52.601818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:52.611417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:52.612850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:52.622320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1154/2330] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:54.261933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:54.262570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:54.262570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:54.291562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] MFA Rules not processed for user `9c1d5ae78d2247ee8afbd26bcc1a79b3`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:54.292804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:54.302579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:54.369652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1177/2331] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:54.374348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:54.385132 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba7825-072c-4239-9e8c-f0551f2b1188 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:54.390038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:54.390725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:54.391013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:54.391296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:54.392005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:54.393954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:54.578224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1155/2332] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:02:54 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:54.579898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:54.595522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875bb95-8024-4e9c-a97a-602894194840 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:54.659849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:54.660582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:54.660886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:54.661178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:54.661891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:54.663955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:54.759976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1178/2333] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:02:54 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:54.765018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:54.775362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190cc49-bd48-46e8-a756-ddd6d05c3aaf admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:54.781144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:54.781144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:54.781353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:54.781662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:54.782394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:54.784011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:54.796463 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1156/2334] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:02:54.800738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:54.811151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a619751-8cfd-479d-a092-c45ad9c99311 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:54.815733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:54.816609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:54.816896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:54.817174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] PATH_INFO: `/v3/projects/74b8587026814352bee325103e906e62/users/4382ef290f444da193de9c31a16e5857/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:54.831760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b8587026814352bee325103e906e62, user_id=4382ef290f444da193de9c31a16e5857, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:54.834426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:54.963495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1179/2335] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:54 2022] PUT /identity/v3/projects/74b8587026814352bee325103e906e62/users/4382ef290f444da193de9c31a16e5857/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:54.968536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:54.978534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149692f-a079-4941-8905-60432912cbf7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:55.035103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:55.035770 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:55.036060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:55.036340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:55.037066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:55.038686 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47ce3d-dc8c-49e3-8876-df5b53f5b01d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:55.048641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1157/2336] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:02:55.053068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:55.064779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0bc8a-d0cd-41bd-b9db-dccf6259c53f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:55.069592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:55.070486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:55.070814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:55.071060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] PATH_INFO: `/v3/projects/74b8587026814352bee325103e906e62/users/4382ef290f444da193de9c31a16e5857/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:55.073106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b8587026814352bee325103e906e62, user_id=4382ef290f444da193de9c31a16e5857, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:55.076331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:55.086842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1180/2337] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:02:55 2022] PUT /identity/v3/projects/74b8587026814352bee325103e906e62/users/4382ef290f444da193de9c31a16e5857/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:56.772590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:56.773136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:56.773264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:56.801206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] MFA Rules not processed for user `4382ef290f444da193de9c31a16e5857`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:56.802510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:56.806738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:56.875976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1158/2338] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:56.889739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:56.892609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:56.956296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b508392-94a3-4c37-bdd8-d4868e725f5f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:56.959919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:56.960577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:56.960863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:56.961199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:56.974454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:57.021479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:57.022982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:57.032606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1181/2339] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:00.084896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:00.095464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a4025-0e73-4c18-add3-5e7eb06f5877 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:00.099117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:00.099766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:00.100091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:00.100417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:00.111960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:00.113413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:00.123472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1159/2340] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:07.811919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=20 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:10.118986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=33 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:21.537637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:21.542531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:21.614755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc375-2031-43ea-a772-b863b781614a service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:21.620124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:21.621516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:21.621899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:21.622268 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:21.624204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:21.635971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:21.638159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:21.648418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1182/2341] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:22.565000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:22.629757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7c9a-f0fb-4ebe-a5f8-9b7d3f7feb74 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:22.634280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:22.644259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:22.644259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:22.644259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] PATH_INFO: `/v3/users/9c1d5ae78d2247ee8afbd26bcc1a79b3` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:22.644259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1d5ae78d2247ee8afbd26bcc1a79b3)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:22.644259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:22.956668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1d5ae78d2247ee8afbd26bcc1a79b3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:22.963489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1d5ae78d2247ee8afbd26bcc1a79b3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:22.964173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1d5ae78d2247ee8afbd26bcc1a79b3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:22.976570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1d5ae78d2247ee8afbd26bcc1a79b3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.104427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-77a7dce4-ca6a-47cc-917c-fd0f4471c9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1d5ae78d2247ee8afbd26bcc1a79b3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.110797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1160/2342] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:22 2022] DELETE /identity/v3/users/9c1d5ae78d2247ee8afbd26bcc1a79b3 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:23.436238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:23.447107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c23a-9b8f-443f-bca7-c873ea4c7cc8 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:23.515509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:23.533683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:23.533683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:23.533683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] PATH_INFO: `/v3/projects/1f74ce84ddcc4403b96438e1cc8ce523` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:23.533683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f74ce84ddcc4403b96438e1cc8ce523)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:23.533683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:23.678141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:23.678591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:23.678736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:23.696803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:23.698011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:23.717761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f74ce84ddcc4403b96438e1cc8ce523', 'request_id': 'req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.754866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1161/2343] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:23.811004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f74ce84ddcc4403b96438e1cc8ce523', 'request_id': 'req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.817125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f74ce84ddcc4403b96438e1cc8ce523', 'request_id': 'req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.829699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] The token cache is being invalidate because project 1f74ce84ddcc4403b96438e1cc8ce523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:23.829699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.830187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:23.831842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1183/2344] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:23 2022] DELETE /identity/v3/projects/1f74ce84ddcc4403b96438e1cc8ce523 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:23.836049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:23.837464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:23.851803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2aa83-bb0d-4833-9d3b-b320c04389ea None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:23.872019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:23.908204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:23.909142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:23.909424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:23.909696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] PATH_INFO: `/v3/users/4382ef290f444da193de9c31a16e5857` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:23.910914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4382ef290f444da193de9c31a16e5857)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:23.912828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:23.925996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:23.927506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:23.951035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:23.996337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edabd05-dc51-47dc-b4bf-fdbe5101b90f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:24.001508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:24.002152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:24.002436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:24.002749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:24.004614 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:24.061527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:24.063157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:24.071469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1184/2345] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:03:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 15:03:24.181195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4382ef290f444da193de9c31a16e5857'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:24.188564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4382ef290f444da193de9c31a16e5857'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:24.189214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4382ef290f444da193de9c31a16e5857'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:24.202010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4382ef290f444da193de9c31a16e5857'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:24.336427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7534433-7f02-44a3-8a6e-60dfa3358d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4382ef290f444da193de9c31a16e5857'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:24.343058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1162/2346] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:23 2022] DELETE /identity/v3/users/4382ef290f444da193de9c31a16e5857 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:24.755207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:24.756637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:24.771041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299799b-5831-4d62-80d8-c236ce5ff94e service swift] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:24.834473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:24.835347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:24.835629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:24.835904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] PATH_INFO: `/v3/projects/74b8587026814352bee325103e906e62` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:24.837079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b8587026814352bee325103e906e62)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:24.839062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.014489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b8587026814352bee325103e906e62', 'request_id': 'req-ecee9aeb-3613-4b2a-b0fe-db141c95166a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:25.087552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b8587026814352bee325103e906e62', 'request_id': 'req-ecee9aeb-3613-4b2a-b0fe-db141c95166a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:25.092606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:25.093859 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b8587026814352bee325103e906e62', 'request_id': 'req-ecee9aeb-3613-4b2a-b0fe-db141c95166a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:25.104904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] The token cache is being invalidate because project 74b8587026814352bee325103e906e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:25.105702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:25.106353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-ecee9aeb-3613-4b2a-b0fe-db141c95166a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:25.107914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1185/2347] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:24 2022] DELETE /identity/v3/projects/74b8587026814352bee325103e906e62 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:25.112818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.113278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.113369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.131803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:25.133002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:25.191298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1163/2348] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:25.200225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.200460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.200741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.219537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:25.220741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:25.229089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1186/2349] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:25.234088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.247300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccfae7-a242-4b33-8262-35d19aed5678 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.249275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.249963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.250256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.250542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.251358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:25.251679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.253335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.265417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1164/2350] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:25.269685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.279689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5c119-6cd6-442f-b8ae-dbfeb1b29324 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.284393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.285035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.287286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.287286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.287286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.288357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.393257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1187/2351] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:03:25 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:25.394548 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.406024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2504681-d1bc-4568-b195-60421fd05287 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.437505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:25.465051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.465712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.465992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.466262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.467295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.469051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.605909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1165/2352] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:03:25 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:25.607196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.617431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1167d-3c7e-418f-9036-8ea323994052 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.622056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.628136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.628136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.628136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.628136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.628136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.640098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1188/2353] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:25.641663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.651407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beddae-069e-413e-ab33-6adfb93b26c2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.656648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.657306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.657795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.658078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.658868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.660452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f1fc2-852f-41a3-8289-4963542d0824 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.670895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1166/2354] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:25.675306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.685684 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47ba9-1fb5-4cd7-9657-4d6b4b3e6234 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:25.690416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:25.691334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.691619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.691897 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] PATH_INFO: `/v3/projects/ce8996b705f24880a8f36e18c84d32ee/users/0ffa860fb8d2472c804379afe5155f58/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.708947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8996b705f24880a8f36e18c84d32ee, user_id=0ffa860fb8d2472c804379afe5155f58, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:25.711642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:25.830808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1189/2355] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:25 2022] PUT /identity/v3/projects/ce8996b705f24880a8f36e18c84d32ee/users/0ffa860fb8d2472c804379afe5155f58/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:03:25.841602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:25.841901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:25.842188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:25.871775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] MFA Rules not processed for user `0ffa860fb8d2472c804379afe5155f58`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:25.873006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:25.937310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1167/2356] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:25.948032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:25.950257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:26.008666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af203bf9-a951-4bf2-9beb-e7b8c8e8c68c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:26.012428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:26.013072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:26.013370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:26.013655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:26.022484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:26.024057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:26.033911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1190/2357] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:34.341912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:34.344406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:34.415860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb8552-8ab9-4a4d-b555-3bb67381a2b4 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:34.421204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:34.429436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:34.429436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:34.429436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] PATH_INFO: `/v3/users/479781403bb94b9788fd589f838fea3f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:34.429436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=479781403bb94b9788fd589f838fea3f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:34.429436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:34.702537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:34.707417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '479781403bb94b9788fd589f838fea3f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:34.715161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '479781403bb94b9788fd589f838fea3f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:34.715876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '479781403bb94b9788fd589f838fea3f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:34.727615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '479781403bb94b9788fd589f838fea3f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:34.827705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '479781403bb94b9788fd589f838fea3f'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:34.834179 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1168/2358] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:34 2022] DELETE /identity/v3/users/479781403bb94b9788fd589f838fea3f => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:35.100639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:35.113284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6498d8-62c5-4879-9609-daf1a041fcb7 service glance] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:35.176820 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:35.177714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:35.178004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:35.178285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] PATH_INFO: `/v3/projects/bb0ee163dc0c4237aaa4e2efb85878fc` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:35.179519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0ee163dc0c4237aaa4e2efb85878fc)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:35.181728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:35.287686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:35.303119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0ee163dc0c4237aaa4e2efb85878fc', 'request_id': 'req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.365070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0ee163dc0c4237aaa4e2efb85878fc', 'request_id': 'req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.372077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0ee163dc0c4237aaa4e2efb85878fc', 'request_id': 'req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.383765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] The token cache is being invalidate because project bb0ee163dc0c4237aaa4e2efb85878fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:35.384125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.384778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.389453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1191/2359] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:35 2022] DELETE /identity/v3/projects/bb0ee163dc0c4237aaa4e2efb85878fc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:35.390752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:35.392286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:35.405506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7b61-d577-4a8a-ab4d-54b9196b7e69 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:35.463680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:35.464556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:35.464841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:35.465112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] PATH_INFO: `/v3/users/a9767369aefb4b1684c6aad00190f675` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:35.466351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9767369aefb4b1684c6aad00190f675)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:35.468355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:35.680031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9767369aefb4b1684c6aad00190f675'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.688094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9767369aefb4b1684c6aad00190f675'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.688763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9767369aefb4b1684c6aad00190f675'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.701131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9767369aefb4b1684c6aad00190f675'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.778982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9767369aefb4b1684c6aad00190f675'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:35.785492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1169/2360] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:35 2022] DELETE /identity/v3/users/a9767369aefb4b1684c6aad00190f675 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:36.051429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:36.051429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:36.067616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c0a02-bcaf-4ce2-aa0f-1ac980bdd681 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:36.127922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:36.131498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:36.131498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:36.131498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] PATH_INFO: `/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:36.131498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4acf7c7eb4248df881d3f014ec44bf6)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:36.132547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:36.236138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4acf7c7eb4248df881d3f014ec44bf6', 'request_id': 'req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:36.302196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4acf7c7eb4248df881d3f014ec44bf6', 'request_id': 'req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:36.308595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4acf7c7eb4248df881d3f014ec44bf6', 'request_id': 'req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:36.322445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] The token cache is being invalidate because project a4acf7c7eb4248df881d3f014ec44bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:36.322747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:36.323456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6f6cdc62-c067-4978-82fe-4c6e4c247d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:36.325642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1192/2361] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:36 2022] DELETE /identity/v3/projects/a4acf7c7eb4248df881d3f014ec44bf6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:45.781684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:46.319002 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=63,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:53.126184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:53.127606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:53.140273 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec122a-9fbf-4d86-9563-b020d25aef92 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:53.212329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:53.213203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:53.213493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:53.213765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] PATH_INFO: `/v3/users/0ffa860fb8d2472c804379afe5155f58` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:53.214993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffa860fb8d2472c804379afe5155f58)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:53.217452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:53.667930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffa860fb8d2472c804379afe5155f58'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:53.675891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffa860fb8d2472c804379afe5155f58'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:53.676670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffa860fb8d2472c804379afe5155f58'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:53.689859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffa860fb8d2472c804379afe5155f58'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:53.868129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffa860fb8d2472c804379afe5155f58'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:53.875103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1170/2362] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:53 2022] DELETE /identity/v3/users/0ffa860fb8d2472c804379afe5155f58 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:53.879898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:53.880406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:53.880518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:53.898776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:53.899995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:53.958461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1193/2363] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:53.968465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:53.969903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:54.029087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdfd43-fa0e-4166-8d7a-656537bd5a9f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:54.032725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:54.033369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:54.033658 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:54.033941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:54.043067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:54.044703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a44247-a58b-4b79-a84c-3b90b8318ab8 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:54.054210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1171/2364] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:03:54.652931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:54.654453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:54.671030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48671e37-994c-4c9e-b633-75752e7d8651 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:54.672654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:54.673521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:54.673806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:54.674082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] PATH_INFO: `/v3/projects/ce8996b705f24880a8f36e18c84d32ee` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:54.675359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8996b705f24880a8f36e18c84d32ee)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:54.677451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:54.854023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8996b705f24880a8f36e18c84d32ee', 'request_id': 'req-cd3af077-654a-4fde-a1f0-e10ea3bc9177'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:54.944714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8996b705f24880a8f36e18c84d32ee', 'request_id': 'req-cd3af077-654a-4fde-a1f0-e10ea3bc9177'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:54.950918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8996b705f24880a8f36e18c84d32ee', 'request_id': 'req-cd3af077-654a-4fde-a1f0-e10ea3bc9177'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:54.962041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] The token cache is being invalidate because project ce8996b705f24880a8f36e18c84d32ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:54.962333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:54.963011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cd3af077-654a-4fde-a1f0-e10ea3bc9177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:54.964566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1194/2365] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:54 2022] DELETE /identity/v3/projects/ce8996b705f24880a8f36e18c84d32ee => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:54.969450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:54.969956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:54.970072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:54.988750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:54.989947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:55.007354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:55.049699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1172/2366] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:55.057733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.058025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.058336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.077283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:55.078500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:55.086549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1195/2367] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:55.090998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.100909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd09e9-ce01-4903-9207-19bda488943a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.105640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.106829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.107070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.114604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.114604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:55.114604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.114604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.121537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1173/2368] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:55.126616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.135888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5476738-fd9e-47db-a077-05b7dee34df5 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.140595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.141244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.141542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.141878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.142612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.144564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.260764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1196/2369] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:03:55 2022] POST /identity/v3/projects => generated 359 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:55.265256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.275837 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c220aaf-7eea-480b-807e-afaac156bff5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.338580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.339200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.339512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.339884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.341039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.343125 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.445554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1174/2370] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:03:55 2022] POST /identity/v3/users => generated 280 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:55.449986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.461406 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7196d2-5d22-40ac-a929-21c1078147d6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.467085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.467851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.468140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.468424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.469157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.470808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.483043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1197/2371] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:55.487117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.497389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d95d-2ea1-48ef-844b-a36436557d0a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.501997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.502684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.502979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.503263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.503997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.505584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da81c-84dc-4e74-b411-4aabd3175600 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.515751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1175/2372] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:55.520035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.531177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968a787-96f7-4dbe-adf5-35242fed5b34 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.536161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.537082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.537366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.537653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] PATH_INFO: `/v3/projects/99ad391d069d4fa6aa76232ddb777c22/users/a78d02d02f1d45eaa715ba384b21019b/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.552780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ad391d069d4fa6aa76232ddb777c22, user_id=a78d02d02f1d45eaa715ba384b21019b, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.555609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.637050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1198/2373] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:55 2022] PUT /identity/v3/projects/99ad391d069d4fa6aa76232ddb777c22/users/a78d02d02f1d45eaa715ba384b21019b/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:55.648735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.649165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.649326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.679740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] MFA Rules not processed for user `a78d02d02f1d45eaa715ba384b21019b`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:55.680981 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:55.755501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1176/2374] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:55.795396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.797814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:55.826456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:55.863589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae44c2-7870-42db-acc3-bbb308636ba8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.867307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.867959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.868273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.868583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.877133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.878710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723157b2-465e-4e86-854f-59064bb5ffb7 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:55.890654 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1199/2375] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:55 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:03:55.901808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:55.921826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:55.964771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbfc85-c360-4a16-b21d-39e09cbdf571 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:55.969993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:55.970896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:55.971183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:55.971464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] PATH_INFO: `/v3/users/a78d02d02f1d45eaa715ba384b21019b` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:55.972667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78d02d02f1d45eaa715ba384b21019b)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:55.974600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:56.232591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78d02d02f1d45eaa715ba384b21019b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:56.239531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78d02d02f1d45eaa715ba384b21019b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:56.240189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78d02d02f1d45eaa715ba384b21019b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:56.252532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78d02d02f1d45eaa715ba384b21019b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:56.365366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78d02d02f1d45eaa715ba384b21019b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:56.372721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1177/2376] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:03:55 2022] DELETE /identity/v3/users/a78d02d02f1d45eaa715ba384b21019b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:56.377749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:56.378247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:56.378360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:56.398741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:56.400005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:56.466842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1200/2377] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:56.476613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:56.478039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:56.535805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94afd09-f90f-4b04-bed9-2a0094328fe6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:56.539409 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:56.540048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:56.540333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:56.540608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:56.551962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:56.553745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367d8c3-b95a-4c0e-8b54-f54288a5dbeb service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:56.564564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1178/2378] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:57.012670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.017936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:57.027109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb204c6d-7adb-40b0-a151-1606683e60b3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.035251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.036134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.036416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.036701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] PATH_INFO: `/v3/projects/99ad391d069d4fa6aa76232ddb777c22` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.038062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ad391d069d4fa6aa76232ddb777c22)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.040120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.177113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ad391d069d4fa6aa76232ddb777c22', 'request_id': 'req-a960ab7f-e060-403e-9980-1977b6e6222a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:57.333621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ad391d069d4fa6aa76232ddb777c22', 'request_id': 'req-a960ab7f-e060-403e-9980-1977b6e6222a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:57.339836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ad391d069d4fa6aa76232ddb777c22', 'request_id': 'req-a960ab7f-e060-403e-9980-1977b6e6222a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:57.351589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] The token cache is being invalidate because project 99ad391d069d4fa6aa76232ddb777c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:57.362446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:57.362446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a960ab7f-e060-403e-9980-1977b6e6222a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:57.362446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1201/2379] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:03:57 2022] DELETE /identity/v3/projects/99ad391d069d4fa6aa76232ddb777c22 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:57.362836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.362836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.362836 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.377342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:57.378915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:57.413100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:57.439321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1179/2380] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:57.447387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.447883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.447981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.467779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:57.468975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:57.476973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1202/2381] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:57.481456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.491371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f001d-72f2-4864-9eca-1a626d39b811 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.495988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.497158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.497480 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.497892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.498753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:57.499117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.500721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.513519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1180/2382] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:57.517787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.526993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4967b-314e-4d8b-b2c2-458da73d548a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.531758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.532401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.532686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.532963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.533658 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.535607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.685127 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1203/2383] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:03:57 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:57.690275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.701039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433b841-2c9c-4bee-974b-f2f0ac1e8134 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.757572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041690-6638-4265-bc79-49a060515292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.758242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041690-6638-4265-bc79-49a060515292 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.758531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041690-6638-4265-bc79-49a060515292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.758848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85041690-6638-4265-bc79-49a060515292 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.759545 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041690-6638-4265-bc79-49a060515292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.761491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85041690-6638-4265-bc79-49a060515292 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.870322 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1181/2384] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:03:57 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:57.875047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.885747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2ed6-f54a-42ea-8586-7ea6be316452 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.890334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.891028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.891313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.891593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.892317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.893921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.904470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1204/2385] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:57.908348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041690-6638-4265-bc79-49a060515292 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.918430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85041690-6638-4265-bc79-49a060515292 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.925591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.925591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.925591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.925591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.925985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.927184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccf448-5a47-4483-94b0-e93373683823 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:57.938683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1182/2386] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:57.942934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:57.952925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d2e39-f2cf-44cb-839f-cb7ce2378447 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:57.957491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:57.958363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:57.958669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:57.958945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] PATH_INFO: `/v3/projects/7886ffbbafa14396b4c805cdb5daaae7/users/55591b7b77be4a17ab1128762f2f712d/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:57.961107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:57.973433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7886ffbbafa14396b4c805cdb5daaae7, user_id=55591b7b77be4a17ab1128762f2f712d, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:57.976180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.067765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1205/2387] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:57 2022] PUT /identity/v3/projects/7886ffbbafa14396b4c805cdb5daaae7/users/55591b7b77be4a17ab1128762f2f712d/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:58.079092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.079494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.080430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.114403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] MFA Rules not processed for user `55591b7b77be4a17ab1128762f2f712d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:58.116229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:58.202281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1183/2388] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:58.207059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.268386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3841c6-6bbe-4bc0-b35e-18c923cbe27f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.273214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.273867 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.274149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.274426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.275172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.277079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.398479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1206/2389] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:03:58 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:03:58.402625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.413992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b38cf-deca-4617-b6d6-b78da655cbfc None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.448195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:58.472226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.472899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.473189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.473468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.474164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.476106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.597697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1184/2390] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:03:58 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:03:58.605134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.616381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aa19-1ed6-4d71-b47e-13223dd7d0ce admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.622035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.622736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.623021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.623297 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.624042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.625629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.636389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1207/2391] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:58.644980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.655756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98095e-a571-46f3-aeea-efe008818ee0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.660324 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.660976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.661262 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.661544 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.662271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.663921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57fae8-e011-4680-824a-0d35fc5cf620 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.673967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1185/2392] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:58.678230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.688314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1482e0-2b1e-4f13-ae20-fd192ed16236 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.693086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.693948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.694230 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.694499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] PATH_INFO: `/v3/projects/e0ce5f400fa54707b054d7d06577d69f/users/1f6776cc8fef41989518253024e58c3e/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.710371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ce5f400fa54707b054d7d06577d69f, user_id=1f6776cc8fef41989518253024e58c3e, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.713309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.846018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1208/2393] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:58 2022] PUT /identity/v3/projects/e0ce5f400fa54707b054d7d06577d69f/users/1f6776cc8fef41989518253024e58c3e/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:58.858310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.858605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.859000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.886468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] MFA Rules not processed for user `1f6776cc8fef41989518253024e58c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:58.888578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:58.955455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1186/2394] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:58.960109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.024042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb407890-0225-40f2-b5cd-d7702c268acf admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.028723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.029380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.029666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.029940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.030628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.032573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.192779 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1209/2395] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:03:58 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:59.197234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.208660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853404fe-7cd9-40ac-8942-cf041e390a5c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.269323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.269919 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.270205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.270483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.271207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.273149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.396331 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1187/2396] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:59.400203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.409215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fd51d-7f54-4b91-aa6e-0434d74f1b59 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.413817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.414483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.414799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.415095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.415840 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.417459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.429149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1210/2397] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:59.433667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.444530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6494df-b5fb-49c4-9ba1-bbc1304debc1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.449075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.450016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.450307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.450590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] PATH_INFO: `/v3/projects/083b4b4083944d7594b9ad0c1081f64a/users/07c8cdfb40d646199fa2b40f58764027/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.465024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=083b4b4083944d7594b9ad0c1081f64a, user_id=07c8cdfb40d646199fa2b40f58764027, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.467756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.577192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1188/2398] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:59 2022] PUT /identity/v3/projects/083b4b4083944d7594b9ad0c1081f64a/users/07c8cdfb40d646199fa2b40f58764027/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:59.581359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.591563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97713f2-b929-4881-b729-5354ef83ed50 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.595240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:59.647429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.648104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.648366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.648647 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.649382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.651026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ffc4-5797-4f8a-af3a-b884917321d2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.660673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1211/2399] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:59.665005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.675784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d46352-b97a-4514-944e-2d56b3db8fbd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.680450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.681325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.681610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.681886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] PATH_INFO: `/v3/projects/083b4b4083944d7594b9ad0c1081f64a/users/07c8cdfb40d646199fa2b40f58764027/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.684145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083b4b4083944d7594b9ad0c1081f64a, user_id=07c8cdfb40d646199fa2b40f58764027, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.687477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.690483 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:59.698183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1189/2400] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:03:59 2022] PUT /identity/v3/projects/083b4b4083944d7594b9ad0c1081f64a/users/07c8cdfb40d646199fa2b40f58764027/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:03:59.708757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.709252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.709744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.742720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] MFA Rules not processed for user `07c8cdfb40d646199fa2b40f58764027`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:59.743922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:59.810497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1212/2401] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:59.821847 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.823671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:59.880955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00a33e-b2e6-433a-9662-10edf9f0d132 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.884685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.885323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.885627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.885891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.941146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.942580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a50d67-8d01-4357-a612-1afdf443493a service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.951411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1190/2402] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:03.249124 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:03.259061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59160dea-df9d-47b7-b435-c1741cfa41da None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:03.263129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:03.263887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:03.264273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:03.264669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:03.321060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:03.322603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:03.332655 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1213/2403] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:04.334573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:04.335393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:04.335393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:04.359534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:04.360802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:04.419832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1191/2404] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:04.432039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:04.433907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:04.464761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:04.504761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef533b23-eb02-48ff-98a2-141ddf6d8b4f service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:04.510070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:04.510749 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:04.511034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:04.511327 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:04.519391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:04.520975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:04.529871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1214/2405] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:04:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:04:05.607806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:05.625813 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5a881-4ea5-4670-872d-daf73ec86877 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:05.630116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:05.631361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:05.631759 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:05.632128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:05.651745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:05.654721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e7832-a2c6-4509-9f86-2513a45d5287 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:05.667287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1192/2406] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:13.048435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:13.167710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0f29-dc71-4559-842b-e54ef9179803 service swift] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:13.173326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:13.174225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:13.174513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:13.174832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] PATH_INFO: `/v3/users/55591b7b77be4a17ab1128762f2f712d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:13.176120 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] RBAC: Authorizing `identity:delete_user(user_id=55591b7b77be4a17ab1128762f2f712d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:13.179106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:13.491484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55591b7b77be4a17ab1128762f2f712d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:13.501134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55591b7b77be4a17ab1128762f2f712d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:13.501939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55591b7b77be4a17ab1128762f2f712d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:13.533279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55591b7b77be4a17ab1128762f2f712d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:13.637029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55591b7b77be4a17ab1128762f2f712d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:13.644828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1215/2407] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:04:13 2022] DELETE /identity/v3/users/55591b7b77be4a17ab1128762f2f712d => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:13.650110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:13.650110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:13.650894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:13.675685 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:13.677019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:13.731122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:13.757989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1193/2408] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:13.773149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:13.774685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:13.843913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:13.893765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ee47f-0614-4585-be90-c61facf7094e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:13.900083 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:13.900992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:13.901587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:13.902436 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:13.914454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:13.922759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:13.927828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1216/2409] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:14.454270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:14.455722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:14.482115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9a805-35d8-4ff3-a065-609c0c36ce1b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:14.490563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:14.491618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:14.491992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:14.495009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] PATH_INFO: `/v3/projects/7886ffbbafa14396b4c805cdb5daaae7` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:14.497429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7886ffbbafa14396b4c805cdb5daaae7)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:14.500406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:14.632864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7886ffbbafa14396b4c805cdb5daaae7', 'request_id': 'req-88506b13-9b88-4edf-94e8-39bb43a22409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:14.774379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7886ffbbafa14396b4c805cdb5daaae7', 'request_id': 'req-88506b13-9b88-4edf-94e8-39bb43a22409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:14.786002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7886ffbbafa14396b4c805cdb5daaae7', 'request_id': 'req-88506b13-9b88-4edf-94e8-39bb43a22409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:14.800357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] The token cache is being invalidate because project 7886ffbbafa14396b4c805cdb5daaae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:14.800660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:14.801784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:14.803830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1194/2410] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:04:14 2022] DELETE /identity/v3/projects/7886ffbbafa14396b4c805cdb5daaae7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:14.809104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:14.828082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca5b3-3268-4b47-89b6-a22734d0e2ba service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:14.938079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:14.939760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:14.940242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:14.940978 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] PATH_INFO: `/v3/users/1f6776cc8fef41989518253024e58c3e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:14.942832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6776cc8fef41989518253024e58c3e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:14.945599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:15.218767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6776cc8fef41989518253024e58c3e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:15.227603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6776cc8fef41989518253024e58c3e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:15.228459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6776cc8fef41989518253024e58c3e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:15.242218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6776cc8fef41989518253024e58c3e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:15.944721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6776cc8fef41989518253024e58c3e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:15.950334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1217/2411] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:04:14 2022] DELETE /identity/v3/users/1f6776cc8fef41989518253024e58c3e => generated 0 bytes in 1143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:16.637170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:16.639814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:16.656295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88506b13-9b88-4edf-94e8-39bb43a22409 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:16.756173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:16.757425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:16.757641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:16.757943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] PATH_INFO: `/v3/projects/e0ce5f400fa54707b054d7d06577d69f` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:16.759284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ce5f400fa54707b054d7d06577d69f)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:16.761278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:16.961702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ce5f400fa54707b054d7d06577d69f', 'request_id': 'req-d7520395-dd42-4a70-aad6-2870e261e8bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.116927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ce5f400fa54707b054d7d06577d69f', 'request_id': 'req-d7520395-dd42-4a70-aad6-2870e261e8bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.124435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ce5f400fa54707b054d7d06577d69f', 'request_id': 'req-d7520395-dd42-4a70-aad6-2870e261e8bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.129316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:17.135264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] The token cache is being invalidate because project e0ce5f400fa54707b054d7d06577d69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:17.135559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.136217 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.138411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1195/2412] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:04:16 2022] DELETE /identity/v3/projects/e0ce5f400fa54707b054d7d06577d69f => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:17.142423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:17.143882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:17.157431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca8192-7070-4035-8afc-3920a137ed06 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:17.210892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:17.216885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:17.217775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:17.218063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:17.218339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] PATH_INFO: `/v3/users/07c8cdfb40d646199fa2b40f58764027` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:17.219579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c8cdfb40d646199fa2b40f58764027)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:17.221491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:17.642559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c8cdfb40d646199fa2b40f58764027'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.671719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c8cdfb40d646199fa2b40f58764027'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.671719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c8cdfb40d646199fa2b40f58764027'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.671719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c8cdfb40d646199fa2b40f58764027'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.794628 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-8ca6a0cd-0215-4ea6-81f9-33be052a950b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c8cdfb40d646199fa2b40f58764027'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:17.801056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1218/2413] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:04:17 2022] DELETE /identity/v3/users/07c8cdfb40d646199fa2b40f58764027 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:18.375565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:18.377249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:18.394877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7520395-dd42-4a70-aad6-2870e261e8bd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:18.466391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:18.467366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:18.467684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:18.467961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] PATH_INFO: `/v3/projects/083b4b4083944d7594b9ad0c1081f64a` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:18.469443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=083b4b4083944d7594b9ad0c1081f64a)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:18.471446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:18.632296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083b4b4083944d7594b9ad0c1081f64a', 'request_id': 'req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:18.731986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083b4b4083944d7594b9ad0c1081f64a', 'request_id': 'req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:18.739145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083b4b4083944d7594b9ad0c1081f64a', 'request_id': 'req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:18.750200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] The token cache is being invalidate because project 083b4b4083944d7594b9ad0c1081f64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:18.750492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:18.751176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c7d3cf4-c550-43a7-8c48-a362102a9fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:18.752728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1196/2414] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:04:18 2022] DELETE /identity/v3/projects/083b4b4083944d7594b9ad0c1081f64a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:19.782013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876d10-ecbf-4fee-abcc-a8dfa92498c7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:19.782596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876d10-ecbf-4fee-abcc-a8dfa92498c7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:19.782596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876d10-ecbf-4fee-abcc-a8dfa92498c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:19.811208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-76876d10-ecbf-4fee-abcc-a8dfa92498c7 None None] MFA Rules not processed for user `cd25c0ad8e1c4fa08eca9b6bddeca55d`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:19.811208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-76876d10-ecbf-4fee-abcc-a8dfa92498c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:19.863349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1219/2415] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:25.390965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.391488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.391599 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.392928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1197/2416] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 15:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:25.398629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50302a-c868-460f-9313-413b8535fb08 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.399328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50302a-c868-460f-9313-413b8535fb08 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.399328 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50302a-c868-460f-9313-413b8535fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.417503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-de50302a-c868-460f-9313-413b8535fb08 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:25.418832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-de50302a-c868-460f-9313-413b8535fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.484923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1220/2417] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:25.496760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.498458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.555697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3773d8-9f5f-4bc6-9747-e72a03cf16e8 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.559554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cdeab1ae06e4df9a270a716ae0a1db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.560194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.560482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.560760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.570956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.572391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962149f-8c09-48db-b9bb-27571fcdf7c5 service glance] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.583615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1198/2418] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:25.908176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.908464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.908743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] PATH_INFO: `/` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.910108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1221/2419] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 15:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:25.914803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907bd49-eaeb-4c7d-8a0b-6b95e6ec186c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.915136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907bd49-eaeb-4c7d-8a0b-6b95e6ec186c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.915415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907bd49-eaeb-4c7d-8a0b-6b95e6ec186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.933642 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3907bd49-eaeb-4c7d-8a0b-6b95e6ec186c None None] MFA Rules not processed for user `931dba704d244ccdbb9f7dbadc68d7df`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:25.934878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3907bd49-eaeb-4c7d-8a0b-6b95e6ec186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.960006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:25.988244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1199/2420] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:26.001659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.003495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.017765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc153d-ca97-4b74-a43b-5ef06f2ec11e None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.021296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.021939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.022232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.022509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.030174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.031780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e96ab-1d41-45d6-b1f9-380dadb6640b service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.036685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:26.040758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1222/2421] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:26.130553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.130952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.131300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] PATH_INFO: `/` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.132990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1200/2422] 10.222.0.74 () {54 vars in 921 bytes} [Wed May 25 15:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:26.139308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924620-9233-4fa7-adb9-12860821cd10 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.139613 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924620-9233-4fa7-adb9-12860821cd10 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.139893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924620-9233-4fa7-adb9-12860821cd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.159231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-df924620-9233-4fa7-adb9-12860821cd10 None None] MFA Rules not processed for user `d4beeef3c39c44f39c8c7ee5f13e9623`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:26.160649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-df924620-9233-4fa7-adb9-12860821cd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.170762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1223/2423] 10.222.0.74 () {58 vars in 1018 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:26.182248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.184315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.197943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad162a0-8f87-483b-94cf-5d02daa53670 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.202153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.202815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.203099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.203379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.213349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.214802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.225418 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1201/2424] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:29.918570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.919089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.919196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.937703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:29.938927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:30.000925 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1224/2425] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:30.023973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:30.025984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:30.142558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100c2b4-e959-4733-8672-1f0c810cb8fe service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:30.153336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:30.154005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:30.154665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:30.154665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:30.166010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:30.167162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:30.176957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1202/2426] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:04:34.204545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:34.214319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7c10-6071-49b8-9ec7-5911333811e0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:34.218041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:34.218739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:34.219035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:34.219317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:34.221157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:34.240060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:34.241531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:34.253952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1225/2427] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:40.173184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=48 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:44.248175 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=34 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:45.874212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.874720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.875676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:45.876063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:45.891431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b368-9954-4327-9a9f-48ca9dc8ac1a service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.897072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.897734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.898020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.898293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.900108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:45.934081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.935498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acdfae-f71f-461d-9ff3-1e33b4597e24 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.935733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.939238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.939883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.940174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.940457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.942211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:45.955214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1203/2428] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:45.956662 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.958105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689033-8fce-48ac-9eae-707a767e09fb service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.971092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1226/2429] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:51.214902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.216378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.230905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76951a-03d9-45fe-81ea-782214feb70f service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.236825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.237455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.237737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.238014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.239788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.331471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.333155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.351729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1204/2430] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:52.807345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:52.807832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:52.807931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:52.825518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:52.826756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:52.834422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1227/2431] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 15:04:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:52.847199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:52.858921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f42e-4d4e-478b-b5b1-752567b8b14a service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:52.864038 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:52.864682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:52.864966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:52.865244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:52.873496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:52.875123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:52.883631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1205/2432] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:04:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:04:55.201503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.210855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8153-bf71-404a-a476-bd2874b56a35 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.214388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.215062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.215354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.215633 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.217382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:55.303584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.305070 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.324211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1228/2433] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:02.881090 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=66 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:05.321488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=70 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:27.273245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:27.283206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff78ac-cc74-41db-9e69-3619f2af26bf service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:27.286786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:27.287441 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:27.287740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:27.288028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:27.289745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:27.354449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:27.355948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:27.365590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1206/2434] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:31.170035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:31.171539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:31.187524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e908-5cd8-4491-a2eb-1b1dba156eb3 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:31.189852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:31.190723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:31.191007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:31.191283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] PATH_INFO: `/v3/users/9dba831ebb584de5a0c84f39901d3c32` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:31.192459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dba831ebb584de5a0c84f39901d3c32)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:31.194457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:31.493807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dba831ebb584de5a0c84f39901d3c32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:31.500550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dba831ebb584de5a0c84f39901d3c32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:31.501233 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dba831ebb584de5a0c84f39901d3c32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:31.513550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dba831ebb584de5a0c84f39901d3c32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:31.652536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9eea8aae-bad5-4a87-825a-cb4306b6ade4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dba831ebb584de5a0c84f39901d3c32'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:31.659185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1229/2435] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:31 2022] DELETE /identity/v3/users/9dba831ebb584de5a0c84f39901d3c32 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:31.924434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:31.936404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860bd9-42d7-415e-b9f6-f49217f3e1c0 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:31.995400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:31.996410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:31.996702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:31.996984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] PATH_INFO: `/v3/projects/89a36e186c5f4d3c8b9030d91f6a30f6` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:31.998181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a36e186c5f4d3c8b9030d91f6a30f6)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.000178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.120636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a36e186c5f4d3c8b9030d91f6a30f6', 'request_id': 'req-5468d1ef-e004-4a3e-8589-d8758b4342e4'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:32.225858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a36e186c5f4d3c8b9030d91f6a30f6', 'request_id': 'req-5468d1ef-e004-4a3e-8589-d8758b4342e4'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:32.233531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a36e186c5f4d3c8b9030d91f6a30f6', 'request_id': 'req-5468d1ef-e004-4a3e-8589-d8758b4342e4'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:32.243929 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] The token cache is being invalidate because project 89a36e186c5f4d3c8b9030d91f6a30f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:32.245240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:32.250201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5468d1ef-e004-4a3e-8589-d8758b4342e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:32.250201 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1207/2436] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:31 2022] DELETE /identity/v3/projects/89a36e186c5f4d3c8b9030d91f6a30f6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:32.252892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.253389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.253484 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.271873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:32.273089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:32.331868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1230/2437] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:32.339761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.340053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.340333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.360088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:32.361282 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:32.368953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1208/2438] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:05:32.373709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.383376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec92bc-136d-4935-8d59-272265d7d791 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.388028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.388702 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.388983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.389258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.389977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:05:32.390290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.391914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.403237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1231/2439] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:05:32.407342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.417193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264edb06-da80-4aea-a079-6b7341b99353 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.421939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.422573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.422887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.423164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.423872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.425746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.441707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:32.615989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1209/2440] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:32 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:32.620666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.630889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c25b6-c0b5-4843-b816-7d19e9b369f2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.687524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.688238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.688526 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.688808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.689500 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.691515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.698827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:32.806244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1232/2441] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:32 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:32.811075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.821279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e18a-3001-477a-a886-753f717cfb73 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.826499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.827281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.827471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.827734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.828455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.830024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.840250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1210/2442] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:32.848077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.858018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1954d-80d3-48e4-b3c0-79288d01f370 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.862713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.863366 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.863653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.863933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.864659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.866308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe129ec-f4db-4c41-a856-2cd002bf674a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:32.877331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1233/2443] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:32.881376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:32.891964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2d8c-2a2a-4c33-a201-8e8eafa57247 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:32.896542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:32.897579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:32.897863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:32.898140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] PATH_INFO: `/v3/projects/22b9e0d82c014a40b4828b19bff22186/users/0ac86e4832d340f298773b5c4cf7feb0/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:32.912650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b9e0d82c014a40b4828b19bff22186, user_id=0ac86e4832d340f298773b5c4cf7feb0, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:32.915678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:33.066882 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1211/2444] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:32 2022] PUT /identity/v3/projects/22b9e0d82c014a40b4828b19bff22186/users/0ac86e4832d340f298773b5c4cf7feb0/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:33.077536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:33.077828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:33.078134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:33.104574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] MFA Rules not processed for user `0ac86e4832d340f298773b5c4cf7feb0`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:33.110694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:33.168973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1234/2445] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:33.180122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:33.181815 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:33.255885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18047b6d-8dc6-4c7a-b441-ddad589f4f57 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:33.261793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:33.262443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:33.262760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:33.263045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:33.270809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:33.272398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faedce0-b2f5-40f4-9616-86a3c4201c0c service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:33.281652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1212/2446] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:33.490466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:33.552388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fe85a-e9eb-41f6-9a9d-531fe60bf39d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:33.556869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:33.557723 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:33.558008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:33.558285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] PATH_INFO: `/v3/users/0ac86e4832d340f298773b5c4cf7feb0` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:33.559774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac86e4832d340f298773b5c4cf7feb0)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:33.561709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:33.793916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac86e4832d340f298773b5c4cf7feb0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:33.801542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac86e4832d340f298773b5c4cf7feb0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:33.802471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac86e4832d340f298773b5c4cf7feb0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:33.816797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:33.822465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac86e4832d340f298773b5c4cf7feb0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:33.917659 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac86e4832d340f298773b5c4cf7feb0'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:33.924153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1235/2447] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:33 2022] DELETE /identity/v3/users/0ac86e4832d340f298773b5c4cf7feb0 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:33.931332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:33.931332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:33.931332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:33.946603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:33.948384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:34.015754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1213/2448] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:05:34.025667 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:34.027138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:34.084799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148fb45-edd1-4687-a1e0-f39b387a2221 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:34.088450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:34.089100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:34.089384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:34.089678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:34.099033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:34.100471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f3fd3-68ef-4baf-89f5-8d4321ea48c8 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:34.109899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1236/2449] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:34.587603 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:34.589071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:34.601820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf76c1a-cfc1-477d-9b5f-643b5c05530c None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:34.607241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:34.608501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:34.608792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:34.609074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] PATH_INFO: `/v3/projects/22b9e0d82c014a40b4828b19bff22186` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:34.610287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b9e0d82c014a40b4828b19bff22186)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:34.612255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:34.773159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:34.780373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b9e0d82c014a40b4828b19bff22186', 'request_id': 'req-26cc0482-a89f-4475-a2cb-42fc19b0a195'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:34.892218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b9e0d82c014a40b4828b19bff22186', 'request_id': 'req-26cc0482-a89f-4475-a2cb-42fc19b0a195'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:34.898914 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b9e0d82c014a40b4828b19bff22186', 'request_id': 'req-26cc0482-a89f-4475-a2cb-42fc19b0a195'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:34.909668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] The token cache is being invalidate because project 22b9e0d82c014a40b4828b19bff22186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:34.909957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:34.910590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-26cc0482-a89f-4475-a2cb-42fc19b0a195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:34.912159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1214/2450] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:34 2022] DELETE /identity/v3/projects/22b9e0d82c014a40b4828b19bff22186 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:34.917173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:34.918289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:34.918412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:34.935815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:34.937010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:34.994753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1237/2451] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:35.002650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.002961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.003248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.023515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:35.024740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:35.033631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1215/2452] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:35.037858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.048161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eab1e-3ac1-4e52-8569-6256d663df47 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.052812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.053467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.053756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.054041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.054809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:05:35.055130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.056799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.068549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1238/2453] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:05:35.072764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.082484 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e9d90-dcfe-44ea-b0e9-ed1ceb9eff04 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.087436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.088082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.088368 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.088648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.089860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.091857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.191406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1216/2454] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:35.195511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.206073 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f6c45-b976-47b4-94f8-9d95cadf9f3d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.264215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.264886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.265178 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.265461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.266143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.268100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.404286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1239/2455] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/users => generated 303 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:35.408412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.419187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bd9e1-97e5-496a-a60a-645371d1ee6a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.423648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.424296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.424612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.424991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.425743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.427369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.438208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1217/2456] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:35.442307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.452755 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6547c-5a49-45bf-b19b-52ff1eb72966 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.457400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.458042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.458323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.458599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.459402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.460996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a58138-3970-4758-b1d9-f79486e5b496 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.462236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:35.472074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1240/2457] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:35.480246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.490394 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc26cb-bcff-4092-9c67-68bc2bad57a3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.494980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.496411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.496695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.496973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] PATH_INFO: `/v3/projects/7899b3d118eb415a972735698e1f5fd8/users/3c244409f1664529b9e91dddbae548e8/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.513187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7899b3d118eb415a972735698e1f5fd8, user_id=3c244409f1664529b9e91dddbae548e8, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.515976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.641358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1218/2458] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:35 2022] PUT /identity/v3/projects/7899b3d118eb415a972735698e1f5fd8/users/3c244409f1664529b9e91dddbae548e8/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:35.652003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.652298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.652582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.679434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] MFA Rules not processed for user `3c244409f1664529b9e91dddbae548e8`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:35.694834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:35.743190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1241/2459] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:05:35.748299 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.809143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69391da9-6662-4834-830f-7d1e0a6111a0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:35.813784 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:35.814432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:35.814746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:35.815028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:35.815722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:35.817621 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:35.967637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1219/2460] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/projects => generated 403 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:35.971711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:35.982862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73cd06-f335-4af4-a4c9-fddeb825f45a None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.036986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.037649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.037936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.038205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.039016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.041054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.212463 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1242/2461] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:35 2022] POST /identity/v3/users => generated 308 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:36.216896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.233247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265637-6bc8-4bf9-8fab-703c7dc12e14 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.233247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.233247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.233247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.233700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.234104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.235735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.246454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1220/2462] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:36.250834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.260921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1b5a8-041c-4fcc-982e-3237cb4166e5 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.265606 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.266520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.266844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.267134 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] PATH_INFO: `/v3/projects/200fda5272cc4829a9870a1b50474e09/users/a8a2acf179a64da2be1710e30f268805/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.282903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200fda5272cc4829a9870a1b50474e09, user_id=a8a2acf179a64da2be1710e30f268805, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.285584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.377991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1243/2463] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:36 2022] PUT /identity/v3/projects/200fda5272cc4829a9870a1b50474e09/users/a8a2acf179a64da2be1710e30f268805/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:36.382040 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.392098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598e012-4dff-40d8-a87d-d68b5564deec admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.429974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:36.447899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.448562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.448848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.449127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.449848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.451504 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895a3c1-4d73-4fcb-b315-87492dfd10b1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.461736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1221/2464] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:36.466442 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.476776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7949d-c7ba-4cf4-8a40-7a0a6c191983 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.481885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.482789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.483077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.483403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] PATH_INFO: `/v3/projects/200fda5272cc4829a9870a1b50474e09/users/a8a2acf179a64da2be1710e30f268805/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.485313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=200fda5272cc4829a9870a1b50474e09, user_id=a8a2acf179a64da2be1710e30f268805, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.487971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.498027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1244/2465] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:36 2022] PUT /identity/v3/projects/200fda5272cc4829a9870a1b50474e09/users/a8a2acf179a64da2be1710e30f268805/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:36.509792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.509792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.510006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.537988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] MFA Rules not processed for user `a8a2acf179a64da2be1710e30f268805`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:36.539259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:36.608256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1222/2466] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:36.619088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.620788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:36.662579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:36.678169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80fe27-0af6-4ac3-b380-776880ab5e4c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.681930 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.682593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.682903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.683183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.692547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.693976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121a707-eaa3-4ac7-9cb5-e5081a6784bb service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:36.703780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1245/2467] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:36.859070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:36.914385 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9005b60-7336-4600-9384-cd8ec1c9dc3f None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:36.919077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:36.919943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:36.920247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:36.920541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] PATH_INFO: `/v3/users/3c244409f1664529b9e91dddbae548e8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:36.921886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c244409f1664529b9e91dddbae548e8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:36.924242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:37.185769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c244409f1664529b9e91dddbae548e8'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:37.192532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c244409f1664529b9e91dddbae548e8'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:37.193194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c244409f1664529b9e91dddbae548e8'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:37.198391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:37.205908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c244409f1664529b9e91dddbae548e8'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:37.380281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c244409f1664529b9e91dddbae548e8'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:37.386845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1223/2468] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:36 2022] DELETE /identity/v3/users/3c244409f1664529b9e91dddbae548e8 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:37.391372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:37.391826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:37.391938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:37.409271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:37.410473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:37.469043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1246/2469] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:37.478726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:37.480684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:37.536711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04270968-b60c-4090-b800-9e1bf4cc3930 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:37.540329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:37.540965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:37.541258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:37.541539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:37.551712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:37.553156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:37.563674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1224/2470] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:37.999367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:38.001042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:38.013287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984d5a2-02e9-4355-b9bf-94158f516745 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:38.018050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:38.018948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:38.019237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:38.019518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] PATH_INFO: `/v3/projects/7899b3d118eb415a972735698e1f5fd8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:38.020728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7899b3d118eb415a972735698e1f5fd8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:38.022690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:38.175333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7899b3d118eb415a972735698e1f5fd8', 'request_id': 'req-a8e2c686-4737-469e-8ac3-0244428f08c6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.291982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7899b3d118eb415a972735698e1f5fd8', 'request_id': 'req-a8e2c686-4737-469e-8ac3-0244428f08c6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.300063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7899b3d118eb415a972735698e1f5fd8', 'request_id': 'req-a8e2c686-4737-469e-8ac3-0244428f08c6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.311581 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] The token cache is being invalidate because project 7899b3d118eb415a972735698e1f5fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:38.311872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.312527 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.314228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1247/2471] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:37 2022] DELETE /identity/v3/projects/7899b3d118eb415a972735698e1f5fd8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:38.318411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:38.328910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90dca-ad9e-4773-98ca-211737fdbb6b service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:38.384194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:38.385183 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:38.385457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:38.385729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] PATH_INFO: `/v3/users/a8a2acf179a64da2be1710e30f268805` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:38.387456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a2acf179a64da2be1710e30f268805)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:38.389401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:38.648886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a2acf179a64da2be1710e30f268805'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.655752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a2acf179a64da2be1710e30f268805'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.656416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a2acf179a64da2be1710e30f268805'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.668113 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a2acf179a64da2be1710e30f268805'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.760980 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-05795444-9882-4965-887d-fa4f0688f6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a2acf179a64da2be1710e30f268805'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:38.768911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1225/2472] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:38 2022] DELETE /identity/v3/users/a8a2acf179a64da2be1710e30f268805 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:39.270799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:39.272563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:39.287845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2c686-4737-469e-8ac3-0244428f08c6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:39.347677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:39.348543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.348828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.349143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] PATH_INFO: `/v3/projects/200fda5272cc4829a9870a1b50474e09` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.350290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200fda5272cc4829a9870a1b50474e09)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:39.352263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:39.359746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:39.512157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200fda5272cc4829a9870a1b50474e09', 'request_id': 'req-4e613049-c14e-4067-a658-3565779655a6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:39.589534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200fda5272cc4829a9870a1b50474e09', 'request_id': 'req-4e613049-c14e-4067-a658-3565779655a6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:39.595657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200fda5272cc4829a9870a1b50474e09', 'request_id': 'req-4e613049-c14e-4067-a658-3565779655a6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:39.608260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] The token cache is being invalidate because project 200fda5272cc4829a9870a1b50474e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:39.608545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:39.609188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4e613049-c14e-4067-a658-3565779655a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:39.610762 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1248/2473] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:39 2022] DELETE /identity/v3/projects/200fda5272cc4829a9870a1b50474e09 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:39.615613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.616106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.616245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.623568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:39.635840 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:39.639272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:39.695077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1226/2474] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:39.702902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.703205 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.703494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.721580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:39.722801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:39.731191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1249/2475] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:39.735578 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:39.745388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d090386-bcd6-49d5-b53e-81d51a337733 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:39.750585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:39.751268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.751556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.751838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.752595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:05:39.752909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:39.754506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:39.765817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1227/2476] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:05:39.770190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:39.780046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a02b0-0fb7-4440-84e3-6db18f3c3144 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:39.784781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:39.785431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.785717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.786000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.786804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:39.788788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:39.905370 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1250/2477] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:39 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:39.909617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:39.921258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff4d8-a783-458c-9f0c-86c4099372e9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:39.980767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:39.981440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:39.981730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:39.982014 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:39.982743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:39.984667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.093700 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1228/2478] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:39 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:40.097693 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.108673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e909eb-0be3-467a-aa96-6ed5c207025c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.113263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.113913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.114197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.114473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.115227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.116819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.128726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1251/2479] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:40.132833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.143574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b289b-427a-42f9-b502-86354ce1d212 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.148397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.149082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.149370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.149655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.150400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.152585 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd270f-c190-4bb2-a901-56b0eb2aa38b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.167121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1229/2480] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:40.168807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.179176 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d2433-2cf2-4983-bfb2-aef9f8949e94 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.184260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.185161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.185446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.185724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] PATH_INFO: `/v3/projects/66407149f284467b86368c0b5e0509b1/users/8a495a69e3a04a4db7d6fe5379d4aa8d/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.201600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=66407149f284467b86368c0b5e0509b1, user_id=8a495a69e3a04a4db7d6fe5379d4aa8d, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.204352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.299055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1252/2481] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:40 2022] PUT /identity/v3/projects/66407149f284467b86368c0b5e0509b1/users/8a495a69e3a04a4db7d6fe5379d4aa8d/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:05:40.310714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.311010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.311294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.338180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] MFA Rules not processed for user `8a495a69e3a04a4db7d6fe5379d4aa8d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:40.340192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:40.399150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:40.404860 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1230/2482] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:40.409355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.470798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b27efc-e0bd-44d2-84ae-82b47fc6d2de admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.475587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.476267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.476551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.476829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.477517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.479572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.605607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1253/2483] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:40 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:40.609669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.620344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43c4d-2c4f-449a-857e-acd1dc534a6a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.677704 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.678495 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.678878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.679209 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.680027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.681948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.772126 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1231/2484] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:40 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:40.776117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.786294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c6596-fdd8-4930-aa4a-3b5183aa6e50 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.790987 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.791646 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.791933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.792213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.792939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.794567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.806005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1254/2485] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:40.811272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.821579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f171-21cf-4443-aa7c-bca66656a355 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.826897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.827781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.828071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.828351 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] PATH_INFO: `/v3/projects/7e52de3c854e4b1588a68322bcecd952/users/f1f2d4162dd947b2849f2e86c95f30f5/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.844722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e52de3c854e4b1588a68322bcecd952, user_id=f1f2d4162dd947b2849f2e86c95f30f5, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.847285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.910089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1232/2486] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:40 2022] PUT /identity/v3/projects/7e52de3c854e4b1588a68322bcecd952/users/f1f2d4162dd947b2849f2e86c95f30f5/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:40.914155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:40.920592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:40.924844 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd1df-23d8-45cc-bf0d-c0f96434a191 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:40.982537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:40.983240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:40.983551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:40.983824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:40.984600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:40.986212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a695201-9059-4bdf-98c1-ee94fa14aadc admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:40.996598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1255/2487] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:41.000899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:41.012496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694de7-86a0-4053-8371-56064e64aaca admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:41.017027 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:41.017893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:41.018175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:41.018449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] PATH_INFO: `/v3/projects/7e52de3c854e4b1588a68322bcecd952/users/f1f2d4162dd947b2849f2e86c95f30f5/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:41.020455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e52de3c854e4b1588a68322bcecd952, user_id=f1f2d4162dd947b2849f2e86c95f30f5, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:41.023154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:41.033348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1233/2488] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:40 2022] PUT /identity/v3/projects/7e52de3c854e4b1588a68322bcecd952/users/f1f2d4162dd947b2849f2e86c95f30f5/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:41.055375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:41.055714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:41.056044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:41.083759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] MFA Rules not processed for user `f1f2d4162dd947b2849f2e86c95f30f5`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:41.084989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:41.154263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1256/2489] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:41.164918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:41.167167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:41.235978 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7283c-b3e7-4566-a9ff-55f298161225 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:41.240294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:41.241008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:41.241369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:41.241789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:41.252278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:41.254396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634fd73-f4d4-4ffa-bc66-85e7e4c32471 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:41.265476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1234/2490] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:44.973050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:45.033617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388e6a8-856c-4764-bc70-b62455ec5894 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:45.038143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:45.039029 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:45.039314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:45.039589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] PATH_INFO: `/v3/users/8a495a69e3a04a4db7d6fe5379d4aa8d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:45.040776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a495a69e3a04a4db7d6fe5379d4aa8d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:45.042761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:45.203334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:45.332060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a495a69e3a04a4db7d6fe5379d4aa8d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:45.338514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a495a69e3a04a4db7d6fe5379d4aa8d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:45.339203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a495a69e3a04a4db7d6fe5379d4aa8d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:45.350605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a495a69e3a04a4db7d6fe5379d4aa8d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:45.454518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a495a69e3a04a4db7d6fe5379d4aa8d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:45.461315 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1257/2491] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:44 2022] DELETE /identity/v3/users/8a495a69e3a04a4db7d6fe5379d4aa8d => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:45.466424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:45.469579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:45.469579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:45.485670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:45.486891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:45.532520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:45.544852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1235/2492] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:05:45.555147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:45.556612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:45.613519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b9854-1bff-42a6-90eb-73a366a31e23 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:45.617253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:45.617891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:45.618174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:45.618449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:45.628289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:45.629745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:45.640263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1258/2493] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:46.143543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:46.144321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:46.158039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f937d-d94d-43c7-8ca8-6474c3204576 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:46.162895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:46.163828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:46.164046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:46.164325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] PATH_INFO: `/v3/projects/66407149f284467b86368c0b5e0509b1` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:46.165558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66407149f284467b86368c0b5e0509b1)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:46.167531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:46.349708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66407149f284467b86368c0b5e0509b1', 'request_id': 'req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.472044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66407149f284467b86368c0b5e0509b1', 'request_id': 'req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.478970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66407149f284467b86368c0b5e0509b1', 'request_id': 'req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.489675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] The token cache is being invalidate because project 66407149f284467b86368c0b5e0509b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:46.489962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.490584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.492131 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1236/2494] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:46 2022] DELETE /identity/v3/projects/66407149f284467b86368c0b5e0509b1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:46.496823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:46.506578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656b2dd-ac89-4def-afa4-3f78ca93ccaa service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:46.562218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:46.563150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:46.563426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:46.563742 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] PATH_INFO: `/v3/users/f1f2d4162dd947b2849f2e86c95f30f5` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:46.564902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f2d4162dd947b2849f2e86c95f30f5)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:46.566832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:46.820691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f2d4162dd947b2849f2e86c95f30f5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.822733 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:46.827945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f2d4162dd947b2849f2e86c95f30f5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.828607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f2d4162dd947b2849f2e86c95f30f5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.840732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f2d4162dd947b2849f2e86c95f30f5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.926269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-48d7e99c-5e4b-44b4-960d-b36fc336d482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f2d4162dd947b2849f2e86c95f30f5'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:46.932605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1259/2495] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:46 2022] DELETE /identity/v3/users/f1f2d4162dd947b2849f2e86c95f30f5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:47.422571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:47.424231 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:47.437831 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9aa2-d937-48a5-9919-6e4f8b2c2561 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:47.495736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:47.497184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:47.497467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:47.497744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] PATH_INFO: `/v3/projects/7e52de3c854e4b1588a68322bcecd952` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:47.498963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e52de3c854e4b1588a68322bcecd952)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:47.500872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:47.653083 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e52de3c854e4b1588a68322bcecd952', 'request_id': 'req-a275499d-bf2b-4095-a0e2-defe2223157c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:47.766205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e52de3c854e4b1588a68322bcecd952', 'request_id': 'req-a275499d-bf2b-4095-a0e2-defe2223157c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:47.773029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e52de3c854e4b1588a68322bcecd952', 'request_id': 'req-a275499d-bf2b-4095-a0e2-defe2223157c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:47.783447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] The token cache is being invalidate because project 7e52de3c854e4b1588a68322bcecd952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:47.783734 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:47.787594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a275499d-bf2b-4095-a0e2-defe2223157c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:47.787594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1237/2496] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:47 2022] DELETE /identity/v3/projects/7e52de3c854e4b1588a68322bcecd952 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:47.791085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:47.791576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:47.791683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:47.811701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:47.813042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:47.873836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1260/2497] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:47.881408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:47.881705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:47.881995 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:47.894437 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:47.900515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:47.901742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:47.909668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1238/2498] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:47.913819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:47.924991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abc7e4-b1c8-4378-a32e-59818740d5fb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:47.929944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:47.930678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:47.930974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:47.931259 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:47.931984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:05:47.932299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:47.934159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:47.946479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1261/2499] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:05:47.951358 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:47.961154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c888a-7ad2-4a4b-9bf3-451de42e36d2 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:47.965764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:47.971197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:47.971197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:47.971197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:47.971197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:47.971197 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.093473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1239/2500] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:47 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:48.097660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.107914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a16d-442d-4180-881f-c92688c5264b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.162878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.163538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.163826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.164103 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.164856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.166806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.312508 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1262/2501] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:48 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:48.316721 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.327186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264b412-e4fd-44ba-a32f-b634cfeabaff admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.331873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.332540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.332825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.333104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.333834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.335500 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.347169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1240/2502] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:48.351105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.361971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742771b-ede2-4a13-968c-d803f735700e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.366781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.367458 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.367738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.368012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.368728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.370584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c4d66-1f99-48b0-8f75-13fb5d783204 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.381090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1263/2503] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:48.385018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.395114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36059a-59ca-44fe-962e-d92a61838ae4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.399601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.400471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.400755 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.401034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] PATH_INFO: `/v3/projects/ac7603541bcd4a468ae49ad374b63487/users/acfbca0ca5904ec498b18a46e85adea8/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.415846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7603541bcd4a468ae49ad374b63487, user_id=acfbca0ca5904ec498b18a46e85adea8, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.418478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.523795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1241/2504] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:48 2022] PUT /identity/v3/projects/ac7603541bcd4a468ae49ad374b63487/users/acfbca0ca5904ec498b18a46e85adea8/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:48.536754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.537086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.537398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.564968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] MFA Rules not processed for user `acfbca0ca5904ec498b18a46e85adea8`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:48.566183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:48.586715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:48.608267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.609898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:48.643496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1264/2505] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:48.647903 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.674370 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabf4c2-05a8-4aed-93a7-92a6ca216acd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.678257 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.678940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.679226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.679506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.681362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:48.692170 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.693617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.702910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1242/2506] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:05:48.714920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e481209-61bc-48a7-8d11-f39955d28264 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.720535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.721215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.721504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.721789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.722493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.724522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:48.870887 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1265/2507] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:48 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:48.871397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:48.883683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cce668-c902-4201-9f65-f9ff788e4aec service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:48.945315 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:48.946176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:48.946503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:48.946854 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:48.947598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:48.955186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.074691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1243/2508] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:48 2022] POST /identity/v3/users => generated 299 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:49.086228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.092622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3825e96-d929-4f0d-89e1-96dcd724679d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.099491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.100329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.100705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.101250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.102232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.105105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.116280 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1266/2509] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:49.120519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.131693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb0fe3-7b3c-4cb4-921d-5bac708b423c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.136549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.144582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.144582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.144582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] PATH_INFO: `/v3/projects/ef05a3994b1d49ca923c77d2c052984e/users/1b230a378db44706958a3d2b882a5a9e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.153727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef05a3994b1d49ca923c77d2c052984e, user_id=1b230a378db44706958a3d2b882a5a9e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.156446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.383025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1244/2510] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:49 2022] PUT /identity/v3/projects/ef05a3994b1d49ca923c77d2c052984e/users/1b230a378db44706958a3d2b882a5a9e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:49.387156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.397208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eb4b1-d271-4ccc-ab84-1d8c8a508e63 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.453881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.454538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.454854 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.455143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.455864 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.457464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44e9a-bc2c-45f6-8d56-ec81c85e06ea admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.467219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1267/2511] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:49.471577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.482460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a49b3-7e42-4139-a165-87de3ae188ec admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.489489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.490401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.490718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.490999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] PATH_INFO: `/v3/projects/ef05a3994b1d49ca923c77d2c052984e/users/1b230a378db44706958a3d2b882a5a9e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.492954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef05a3994b1d49ca923c77d2c052984e, user_id=1b230a378db44706958a3d2b882a5a9e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.495651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.502002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:49.507293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1245/2512] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:49 2022] PUT /identity/v3/projects/ef05a3994b1d49ca923c77d2c052984e/users/1b230a378db44706958a3d2b882a5a9e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:49.517657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.522663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.522663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.543776 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] MFA Rules not processed for user `1b230a378db44706958a3d2b882a5a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:49.545481 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:49.566345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:49.612771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1268/2513] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:49.624675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.626444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:49.683788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b1725-5486-440b-86e1-72df624b4b1b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.688135 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.688796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.689065 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.689341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.698305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.699760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f45259-e7e2-431a-8313-bb84dbfc5ad4 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:49.709061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1246/2514] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:49.737934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:49.799535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b66d-c442-4564-97b4-3f164ca5c0da None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:49.804219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:49.805089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:49.805415 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:49.805697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] PATH_INFO: `/v3/users/acfbca0ca5904ec498b18a46e85adea8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:49.806971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfbca0ca5904ec498b18a46e85adea8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:49.808899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:50.013218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfbca0ca5904ec498b18a46e85adea8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:50.019938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfbca0ca5904ec498b18a46e85adea8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:50.020596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfbca0ca5904ec498b18a46e85adea8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:50.032862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfbca0ca5904ec498b18a46e85adea8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:50.114595 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfbca0ca5904ec498b18a46e85adea8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:50.120812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1269/2515] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:49 2022] DELETE /identity/v3/users/acfbca0ca5904ec498b18a46e85adea8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:50.125318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:50.125676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:50.125963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:50.143263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:50.144444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:50.201661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1247/2516] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:50.211168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:50.212690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:50.256245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:50.280043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b628f9-9ab2-4946-8c7c-837df724261d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:50.280043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:50.280043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:50.280043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:50.280649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:50.287855 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:50.289293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:50.299213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1270/2517] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:50.751582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:50.753336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:50.765558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6b495-618a-4c61-af10-71d3ddac35c5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:50.770310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:50.771349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:50.771637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:50.772058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] PATH_INFO: `/v3/projects/ac7603541bcd4a468ae49ad374b63487` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:50.773292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7603541bcd4a468ae49ad374b63487)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:50.775279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:50.907812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7603541bcd4a468ae49ad374b63487', 'request_id': 'req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.006481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7603541bcd4a468ae49ad374b63487', 'request_id': 'req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.013899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7603541bcd4a468ae49ad374b63487', 'request_id': 'req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.024376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] The token cache is being invalidate because project ac7603541bcd4a468ae49ad374b63487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:51.024663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.025298 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.026880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1248/2518] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:50 2022] DELETE /identity/v3/projects/ac7603541bcd4a468ae49ad374b63487 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:51.031576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:51.041744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc24c09-186d-4cd2-ba58-6c99df4d9a72 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:51.100156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:51.101024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:51.101374 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:51.101701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] PATH_INFO: `/v3/users/1b230a378db44706958a3d2b882a5a9e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:51.102939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b230a378db44706958a3d2b882a5a9e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:51.104876 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:51.404290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b230a378db44706958a3d2b882a5a9e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.411699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b230a378db44706958a3d2b882a5a9e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.412399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b230a378db44706958a3d2b882a5a9e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.425109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b230a378db44706958a3d2b882a5a9e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.557616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-64009709-3758-495a-ae54-4285f5ad938a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b230a378db44706958a3d2b882a5a9e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:51.565144 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1271/2519] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:05:51 2022] DELETE /identity/v3/users/1b230a378db44706958a3d2b882a5a9e => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:52.104907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.108350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:52.121310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa7d5a-dc66-4bed-99bb-7ddb725e97b6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:52.141605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:52.178532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:52.179414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.179694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.179967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] PATH_INFO: `/v3/projects/ef05a3994b1d49ca923c77d2c052984e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.181142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef05a3994b1d49ca923c77d2c052984e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:52.183228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:52.341639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef05a3994b1d49ca923c77d2c052984e', 'request_id': 'req-095fdb1a-1681-4f2f-acd4-06ec869dc018'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:52.412284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef05a3994b1d49ca923c77d2c052984e', 'request_id': 'req-095fdb1a-1681-4f2f-acd4-06ec869dc018'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:52.419144 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef05a3994b1d49ca923c77d2c052984e', 'request_id': 'req-095fdb1a-1681-4f2f-acd4-06ec869dc018'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:52.430141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] The token cache is being invalidate because project ef05a3994b1d49ca923c77d2c052984e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:05:52.430430 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:52.431084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-095fdb1a-1681-4f2f-acd4-06ec869dc018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:05:52.432623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1249/2520] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:05:52 2022] DELETE /identity/v3/projects/ef05a3994b1d49ca923c77d2c052984e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:52.438177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.438686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.438847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.457401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:52.458615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:52.511395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:52.520523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1272/2521] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:52.528193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.528485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.528767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.547645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:52.548855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:52.557056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1250/2522] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:52.561609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.572333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f9c69-e791-4492-bcd5-0c3ff6aa742b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:52.577811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:52.578498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.578811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.579086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.579819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:05:52.580130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:52.581759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:52.594592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1273/2523] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:05:52.599025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.608383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d717f9f-783c-4e85-a4b1-4b109346c9fa None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:52.613274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:52.613935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.614226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.614507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.615236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:52.617150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:52.768787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1251/2524] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:52 2022] POST /identity/v3/projects => generated 391 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:52.773418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.783354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a1011-6115-4b7d-8222-ae6321af0369 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:52.839248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:52.839932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.840223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.840535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.841271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:52.843322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:52.980912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1274/2525] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:52 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:52.985226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.998982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9f3ac-dadf-490d-bbfb-d722479b1a31 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.004641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.005464 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.005786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.006094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.006915 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.008791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.020844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1252/2526] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:53.024988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.035398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680ac5-287e-42ee-9824-a684a7971c6f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.040388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.041155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.041444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.041722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.042450 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.044087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65986bad-c10a-49f4-99e3-b33dc4779196 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.054967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1275/2527] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:05:53.061515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.071541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8fb42-c739-4637-bee1-03e8f1d8a69c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.076148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.077051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.077337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.077619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] PATH_INFO: `/v3/projects/a34c5cfe3c0a49e9aa92ce5799ebb9ff/users/414f533282d24ed091cfd9d35425797c/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.092314 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34c5cfe3c0a49e9aa92ce5799ebb9ff, user_id=414f533282d24ed091cfd9d35425797c, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.095017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.201206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1253/2528] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:53 2022] PUT /identity/v3/projects/a34c5cfe3c0a49e9aa92ce5799ebb9ff/users/414f533282d24ed091cfd9d35425797c/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:53.212738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.214980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.214980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.240947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] MFA Rules not processed for user `414f533282d24ed091cfd9d35425797c`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:53.242117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:53.311368 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1276/2529] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:53.315873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.371200 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:53.382897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a33d5-766a-44b8-b038-7bcdb5be3024 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.387604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.388258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.388543 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.388849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.389569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.391506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.481579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1254/2530] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:05:53 2022] POST /identity/v3/projects => generated 393 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:05:53.486307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.497126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da33f-34e3-42ab-8600-f1e3e148672d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.546586 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:53.553209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.556787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.556787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.556787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.556787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.557235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.678621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1277/2531] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:05:53 2022] POST /identity/v3/users => generated 303 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:05:53.682607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.693221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba49132-185c-4223-b157-9514aa0370a1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.698025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.698714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.698999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.699281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.700004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.701589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.713160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1255/2532] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:53.717264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.727945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136d9a-a70c-4e1f-924e-08c7a9d56bee admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.732832 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.734356 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.734669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.734941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] PATH_INFO: `/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd/users/464171f14c694c8ca81e103442921b5c/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.750074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbcd8d5b5a44a22a1dab011a18a10fd, user_id=464171f14c694c8ca81e103442921b5c, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.752795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.857852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1278/2533] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:53 2022] PUT /identity/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd/users/464171f14c694c8ca81e103442921b5c/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:53.861809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.872099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86550281-8b9a-4967-a636-1ade5eccb759 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.927572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.928230 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.928512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.928797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.929503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.931111 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66611931-502c-4335-ac1b-7e5c4f490a1c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.941243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1256/2534] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:53.945425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:53.956283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce2e07-7360-4a24-89ff-1738c61fce78 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:53.960922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:53.961795 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.962082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.962389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] PATH_INFO: `/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd/users/464171f14c694c8ca81e103442921b5c/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:53.966219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbcd8d5b5a44a22a1dab011a18a10fd, user_id=464171f14c694c8ca81e103442921b5c, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:53.969523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:53.980126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1279/2535] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:05:53 2022] PUT /identity/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd/users/464171f14c694c8ca81e103442921b5c/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:05:53.990258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:53.990554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:53.990871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:54.017198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] MFA Rules not processed for user `464171f14c694c8ca81e103442921b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:54.018335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:54.086808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1257/2536] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:54.098187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:54.099878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:54.156201 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835bfa-34f8-42f8-a5c0-377dc5466e9e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:54.159944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:54.160615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:54.160904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:54.161187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:54.170269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:54.171757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:54.181424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1280/2537] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:54.227805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:54.229184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:54.243986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cddd6-0279-433a-88f9-33bf16ad3e7b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:54.248244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:54.248887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:54.249166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:54.249476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:54.297925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:54.311309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:54.313417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:54.323063 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1258/2538] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:55.941010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:55.942464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:56.020485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be50f-695b-4eba-8633-d6ee7d9dc6e7 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:56.026023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:56.026707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:56.026992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:56.027273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:56.029277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:56.042022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:56.043738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c65c0-142b-49ed-8c21-ab9c1001d213 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:56.055768 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1281/2539] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:01.080274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:01.143455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae328146-7ce5-411e-9a55-33cbd7f6b236 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:01.148177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:01.149041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:01.149328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:01.149668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] PATH_INFO: `/v3/users/414f533282d24ed091cfd9d35425797c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:01.150884 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414f533282d24ed091cfd9d35425797c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:01.152864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:01.428592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414f533282d24ed091cfd9d35425797c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:01.435936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414f533282d24ed091cfd9d35425797c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:01.436607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414f533282d24ed091cfd9d35425797c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:01.448649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414f533282d24ed091cfd9d35425797c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:01.551876 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414f533282d24ed091cfd9d35425797c'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:01.558392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1259/2540] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:01 2022] DELETE /identity/v3/users/414f533282d24ed091cfd9d35425797c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:01.563378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:01.570089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:01.570089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:01.584198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:01.585455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:01.619789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:01.650168 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1282/2541] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:01.664391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:01.665838 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:01.708025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:01.709549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:01.728346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3722c4-76a7-4d3d-baff-4dcc43cd2c34 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:01.731889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:01.732524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:01.732806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:01.733084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:01.742240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:01.743707 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:01.754382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1260/2542] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:01.766615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c949a-dbc4-442c-9a58-1cec4f17f62c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:01.771163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:01.771816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:01.772101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:01.772379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:01.774272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:01.786610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:01.788115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:01.797977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1283/2543] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:02.356682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:02.358877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:02.377420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb1b88-0f8d-449c-b96a-005b70ec5edc service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:02.383158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:02.389062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:02.389062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:02.389062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] PATH_INFO: `/v3/projects/a34c5cfe3c0a49e9aa92ce5799ebb9ff` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:02.389062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34c5cfe3c0a49e9aa92ce5799ebb9ff)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:02.389062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:02.390584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:02.541532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34c5cfe3c0a49e9aa92ce5799ebb9ff', 'request_id': 'req-0c3e7601-7ff0-4e59-b162-60d9208be758'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.629590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34c5cfe3c0a49e9aa92ce5799ebb9ff', 'request_id': 'req-0c3e7601-7ff0-4e59-b162-60d9208be758'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.636352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34c5cfe3c0a49e9aa92ce5799ebb9ff', 'request_id': 'req-0c3e7601-7ff0-4e59-b162-60d9208be758'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.647528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] The token cache is being invalidate because project a34c5cfe3c0a49e9aa92ce5799ebb9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:02.647817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.648478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.650186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1261/2544] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:02 2022] DELETE /identity/v3/projects/a34c5cfe3c0a49e9aa92ce5799ebb9ff => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:02.654150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:02.663948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2fc21-d404-4ebc-8ac1-6c57b023e825 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:02.720380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:02.721246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:02.721532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:02.721838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] PATH_INFO: `/v3/users/464171f14c694c8ca81e103442921b5c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:02.723063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=464171f14c694c8ca81e103442921b5c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:02.725009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:02.951612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464171f14c694c8ca81e103442921b5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.960096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464171f14c694c8ca81e103442921b5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.960881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464171f14c694c8ca81e103442921b5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:02.975843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464171f14c694c8ca81e103442921b5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:03.060765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-902c3512-9905-497a-b7fe-f86b3651c86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464171f14c694c8ca81e103442921b5c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:03.062429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:03.068616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1284/2545] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:02 2022] DELETE /identity/v3/users/464171f14c694c8ca81e103442921b5c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:03.639534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:03.642107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:03.667634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7601-7ff0-4e59-b162-60d9208be758 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:03.728990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:03.729931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:03.730288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:03.730571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] PATH_INFO: `/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:03.731898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbcd8d5b5a44a22a1dab011a18a10fd)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:03.733842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:03.879785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbcd8d5b5a44a22a1dab011a18a10fd', 'request_id': 'req-ad412281-fc45-43b7-bf58-f181d31c0410'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:03.997333 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbcd8d5b5a44a22a1dab011a18a10fd', 'request_id': 'req-ad412281-fc45-43b7-bf58-f181d31c0410'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:04.005471 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbcd8d5b5a44a22a1dab011a18a10fd', 'request_id': 'req-ad412281-fc45-43b7-bf58-f181d31c0410'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:04.017986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] The token cache is being invalidate because project afbcd8d5b5a44a22a1dab011a18a10fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:04.018406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:04.019728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ad412281-fc45-43b7-bf58-f181d31c0410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:04.021399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1262/2546] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:03 2022] DELETE /identity/v3/projects/afbcd8d5b5a44a22a1dab011a18a10fd => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:04.026326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.026849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.026947 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.046920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:04.048499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:04.113181 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1285/2547] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:04.121309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.121597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.121877 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.141885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:04.143399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:04.152593 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1263/2548] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:04.157217 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.168579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c72e9-9f21-4fd5-a904-4dc8aeda0dd0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.173445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.174114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.174403 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.174722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.175986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:04.176299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.177956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.191557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1286/2549] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:04.196171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.206413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539aed3-7b67-43b9-8ff2-12d7d6aff806 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.211317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.211974 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.212258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.212602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.213420 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.215403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.225905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:04.374690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1264/2550] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:04.379482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.390501 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ceafa-5673-43dd-af0c-bb3b111c5c0c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.451836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.452551 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.452783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.453059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.453747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.455739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.603492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1287/2551] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:04.604618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.616289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e9ec-c0e4-43d2-81fc-109d1cf42e5d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.621134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.621817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.622103 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.622383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.623169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.624786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.636308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1265/2552] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:04.640270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.652496 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794f5b-e8b5-4eba-b982-aec8da33687c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.658106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.659020 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.659836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.660115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.661019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.662952 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970088f8-163f-4f7c-a75c-a9ed11b70dec admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.678035 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1288/2553] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:04.680017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.692213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a980a2e-7d91-4435-9844-f317552b1fe2 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.696911 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.697916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.698203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.698482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] PATH_INFO: `/v3/projects/db753bad917343e691d952bffca34e98/users/367fc3d2631c459eafa13799b8f68905/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.714988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db753bad917343e691d952bffca34e98, user_id=367fc3d2631c459eafa13799b8f68905, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:04.720579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:04.807539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1266/2554] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:04 2022] PUT /identity/v3/projects/db753bad917343e691d952bffca34e98/users/367fc3d2631c459eafa13799b8f68905/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:04.819229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.819520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.819804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.847846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] MFA Rules not processed for user `367fc3d2631c459eafa13799b8f68905`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:04.849098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:04.922434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:04.937666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1289/2555] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:04.942143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.026122 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855b50-d181-452d-8d8a-795b1b2cba17 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.030973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.031636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.031940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.032202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.032905 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.035518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.173906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1267/2556] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:04 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:05.177782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.189535 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950b2c7-dae3-4b96-851d-7df97d5101a0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.248852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.249798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.249798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.250094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.250869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.252796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.341265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1290/2557] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:05 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:05.350743 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.360462 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b240fa-a70d-4c7d-bb53-876f30e3f032 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.365819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.366595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.366984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.367335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.368199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.370964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.385150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1268/2558] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:05.389422 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.402250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d7c4-89dc-4dd7-81c0-d455b23ad3e2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.407758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.408657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.408972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.409309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] PATH_INFO: `/v3/projects/f267278f7aef48a4a637847539805c0d/users/f4f0b7db5f7e405988281fe50a557fc6/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.432623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f267278f7aef48a4a637847539805c0d, user_id=f4f0b7db5f7e405988281fe50a557fc6, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.435734 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.585094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1291/2559] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:05 2022] PUT /identity/v3/projects/f267278f7aef48a4a637847539805c0d/users/f4f0b7db5f7e405988281fe50a557fc6/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:05.589427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.599664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfafc18-91dd-4d90-a762-787b66cd17e5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.631668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.633598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.666486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.667271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.667607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.667943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.668673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.670283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.682447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1269/2560] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:05.687414 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.697862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07a5d-9091-4d55-a5c6-460a8a6b9f68 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.702889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.703941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.704068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8873-4075-4c9f-966e-3cbf701a7a33 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.704276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.704626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] PATH_INFO: `/v3/projects/f267278f7aef48a4a637847539805c0d/users/f4f0b7db5f7e405988281fe50a557fc6/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.706941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f267278f7aef48a4a637847539805c0d, user_id=f4f0b7db5f7e405988281fe50a557fc6, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.709532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.710165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f5fec-3ee3-4281-a95c-1e75736de652 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.712674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.713063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.716270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] PATH_INFO: `/v3/users/237510469a8e4e36ad40a34cdf705656` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.716270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=237510469a8e4e36ad40a34cdf705656)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.716772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.722921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1270/2561] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:05 2022] PUT /identity/v3/projects/f267278f7aef48a4a637847539805c0d/users/f4f0b7db5f7e405988281fe50a557fc6/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:05.733127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.733512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.733866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.759470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:05.762295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] MFA Rules not processed for user `f4f0b7db5f7e405988281fe50a557fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:05.763600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.831119 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1271/2562] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:05.843096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.844413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.900901 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf917e-0a82-4c9b-b303-43030b6afc4d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.904720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.905375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.905641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.905916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.964386 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.966171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.975224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237510469a8e4e36ad40a34cdf705656'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:05.975596 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1272/2563] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:05.983346 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237510469a8e4e36ad40a34cdf705656'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:05.984157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237510469a8e4e36ad40a34cdf705656'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:05.989443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:05.997273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237510469a8e4e36ad40a34cdf705656'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.063584 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6d8d462c-8005-4741-a3ac-f59c307636e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237510469a8e4e36ad40a34cdf705656'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.070895 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1292/2564] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:05 2022] DELETE /identity/v3/users/237510469a8e4e36ad40a34cdf705656 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:06.338162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:06.392011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:06.403177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c059-4b3b-47df-a125-a9877831d9b9 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:06.458913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:06.459765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:06.460051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:06.460334 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] PATH_INFO: `/v3/projects/18c8213736cf4113951fbb3520c18183` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:06.461558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c8213736cf4113951fbb3520c18183)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:06.463656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:06.615226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c8213736cf4113951fbb3520c18183', 'request_id': 'req-48ebb711-e737-45d5-b862-fd08dd268429'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.760799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c8213736cf4113951fbb3520c18183', 'request_id': 'req-48ebb711-e737-45d5-b862-fd08dd268429'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.767878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c8213736cf4113951fbb3520c18183', 'request_id': 'req-48ebb711-e737-45d5-b862-fd08dd268429'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.778595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] The token cache is being invalidate because project 18c8213736cf4113951fbb3520c18183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:06.778933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.779564 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-48ebb711-e737-45d5-b862-fd08dd268429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:06.781107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1273/2565] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:06 2022] DELETE /identity/v3/projects/18c8213736cf4113951fbb3520c18183 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:06.787026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:06.787521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:06.787636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:06.805256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:06.812704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:06.870339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1293/2566] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:06.878948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:06.879304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:06.879562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:06.900259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:06.901509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:06.909655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1274/2567] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:06.913919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:06.923648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18901c-7419-417f-9dc8-278f954d08bb None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:06.928309 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:06.928962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:06.929250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:06.929528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:06.930249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:06.930598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:06.932229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:06.944543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1294/2568] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:06.948955 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:06.959030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62057a0-910c-44d1-88ff-5bade213b983 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:06.964236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:06.964502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:06.964788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:06.965070 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:06.965754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:06.968309 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.059638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1275/2569] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:06 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:07.063912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.074482 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cd30-209a-4cce-9c90-593c4794addc admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.131593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.132185 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.132474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.132758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.133459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.135432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.256085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1295/2570] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:07 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:07.260202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.273261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c519a-352f-4148-abf8-a10ace125e8b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.275326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.275990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.276274 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.276554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.277279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.278902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.289762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1276/2571] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:07.293937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.304918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783190-6985-4330-aa9d-78ff41afc357 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.310025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.310710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.311025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.316213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.316213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.316213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259f817-05d2-44d2-8b13-2d96b78cb2a4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.324873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1296/2572] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:07.329294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.339641 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45879bf0-d07a-44ca-ae06-f78cd2d0cfb5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.344306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.345184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.350705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.350705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] PATH_INFO: `/v3/projects/65f8ef529d654842aeaa7c931de23797/users/acc4fa794e374400b19440d16a0c7521/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.360524 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f8ef529d654842aeaa7c931de23797, user_id=acc4fa794e374400b19440d16a0c7521, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.363407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.592136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1277/2573] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:07 2022] PUT /identity/v3/projects/65f8ef529d654842aeaa7c931de23797/users/acc4fa794e374400b19440d16a0c7521/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:07.602843 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7569f-64a1-458c-be08-82e930914223 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.603140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7569f-64a1-458c-be08-82e930914223 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.603429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7569f-64a1-458c-be08-82e930914223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.632232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-89d7569f-64a1-458c-be08-82e930914223 None None] MFA Rules not processed for user `acc4fa794e374400b19440d16a0c7521`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:07.633557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-89d7569f-64a1-458c-be08-82e930914223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:07.667079 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:07.700236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1297/2574] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:07.704769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.734460 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:07.765623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd7e8-75b6-4e48-81a1-2dec37c65bef admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.770286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.770975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.771261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.771552 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.772238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.774216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:07.896137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1278/2575] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:07 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:07.901060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7569f-64a1-458c-be08-82e930914223 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:07.913605 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7569f-64a1-458c-be08-82e930914223 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:07.968846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:07.969683 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:07.969830 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:07.970112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:07.970878 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:07.972794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.099873 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1298/2576] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:07 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:08.104377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.114745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6928f-8c40-40b8-b39c-5513c7039dbf admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.119413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.120059 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.120342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.120619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.121339 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.123003 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.133318 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1279/2577] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:08.137710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.148441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7aca-f12a-40e4-80b2-6db3aa65ab26 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.153333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.154347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.154746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.155078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] PATH_INFO: `/v3/projects/011bf4c710a7476a8cb9797a85e180fc/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.171232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011bf4c710a7476a8cb9797a85e180fc, user_id=d3a8b0b8b09c49aa9c8d8a306b4b86ca, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.173882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.292471 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1299/2578] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:08 2022] PUT /identity/v3/projects/011bf4c710a7476a8cb9797a85e180fc/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:08.296558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.306513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e0f7c-b8c2-4f5d-aa7a-fe514ce17fc4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.370539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.371233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.371523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.371799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.372520 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.374171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdc053-baf4-4a53-baa4-567fae5f936c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.384381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1280/2579] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:08.388690 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.399637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ed2e-35bd-4640-b66b-61a5e71b6cf2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.404499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.405391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.405677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.405960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] PATH_INFO: `/v3/projects/011bf4c710a7476a8cb9797a85e180fc/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.408039 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011bf4c710a7476a8cb9797a85e180fc, user_id=d3a8b0b8b09c49aa9c8d8a306b4b86ca, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.411048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.421281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1300/2580] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:08 2022] PUT /identity/v3/projects/011bf4c710a7476a8cb9797a85e180fc/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:08.436850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.437136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.437417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.465043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] MFA Rules not processed for user `d3a8b0b8b09c49aa9c8d8a306b4b86ca`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:08.466264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:08.535102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1281/2581] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:08.546489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.548282 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:08.605588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac59a6-78fa-436a-90d6-79129a003fb1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.609610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.610276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.610585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.610900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.620592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.622024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.631255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1301/2582] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:08.867842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:08.878080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c78c7-6f8f-42c3-9331-d5a7cab15f24 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:08.882475 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:08.883176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:08.883508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:08.883864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:08.903213 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:08.941752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:08.943436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffc589-bd49-47d3-90fb-278f56298d60 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:08.954078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1282/2583] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:10.438627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:10.440157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:10.473477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:10.547966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7d15-704c-4d97-90ed-6fed9b9c9d10 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:10.552520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:10.553383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:10.553676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:10.553940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] PATH_INFO: `/v3/users/367fc3d2631c459eafa13799b8f68905` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:10.555237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367fc3d2631c459eafa13799b8f68905)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:10.557524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:10.803728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367fc3d2631c459eafa13799b8f68905'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:10.811454 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367fc3d2631c459eafa13799b8f68905'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:10.812164 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367fc3d2631c459eafa13799b8f68905'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:10.824229 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367fc3d2631c459eafa13799b8f68905'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:10.983599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367fc3d2631c459eafa13799b8f68905'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:10.989955 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1302/2584] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:10 2022] DELETE /identity/v3/users/367fc3d2631c459eafa13799b8f68905 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:10.995305 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:10.995792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:10.995967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:11.013832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:11.017970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:11.075487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1283/2585] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:11.093625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:11.093625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:11.154807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:11.171976 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75614987-b912-4682-b67c-7487ea398e74 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:11.175745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:11.176391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:11.176678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:11.176957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:11.187857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:11.189303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:11.198933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1303/2586] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:11.659325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:11.660791 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:11.673017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1c7c-ce1c-41aa-9e43-476a9ced5c5e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:11.677729 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:11.678569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:11.678948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:11.679237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] PATH_INFO: `/v3/projects/db753bad917343e691d952bffca34e98` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:11.681098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db753bad917343e691d952bffca34e98)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:11.683098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:11.819501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db753bad917343e691d952bffca34e98', 'request_id': 'req-7f40e67e-7d4c-4389-a7e4-e453d09c6308'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:11.894583 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db753bad917343e691d952bffca34e98', 'request_id': 'req-7f40e67e-7d4c-4389-a7e4-e453d09c6308'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:11.901384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db753bad917343e691d952bffca34e98', 'request_id': 'req-7f40e67e-7d4c-4389-a7e4-e453d09c6308'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:11.911761 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] The token cache is being invalidate because project db753bad917343e691d952bffca34e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:11.912045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:11.912691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:11.914212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1284/2587] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:11 2022] DELETE /identity/v3/projects/db753bad917343e691d952bffca34e98 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:11.918810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:11.929411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d2754-a51a-4a97-9487-89cf60b79eb7 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:11.986972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:11.987850 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:11.988140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:11.988421 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] PATH_INFO: `/v3/users/f4f0b7db5f7e405988281fe50a557fc6` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:11.989627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f0b7db5f7e405988281fe50a557fc6)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:11.991604 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:12.257109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f0b7db5f7e405988281fe50a557fc6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:12.257109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f0b7db5f7e405988281fe50a557fc6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:12.257109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f0b7db5f7e405988281fe50a557fc6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:12.259074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f0b7db5f7e405988281fe50a557fc6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:12.400971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6da37576-4f46-46a0-b537-8568dff48142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f0b7db5f7e405988281fe50a557fc6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:12.407512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1304/2588] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:11 2022] DELETE /identity/v3/users/f4f0b7db5f7e405988281fe50a557fc6 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:12.910976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:12.913075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:12.926601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40e67e-7d4c-4389-a7e4-e453d09c6308 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:12.973904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:12.990997 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:12.998389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:12.998389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:12.998389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] PATH_INFO: `/v3/projects/f267278f7aef48a4a637847539805c0d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:12.998389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f267278f7aef48a4a637847539805c0d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:12.998389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.163069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f267278f7aef48a4a637847539805c0d', 'request_id': 'req-874ae9e3-1194-442e-a3bf-a24288429beb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:13.317747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f267278f7aef48a4a637847539805c0d', 'request_id': 'req-874ae9e3-1194-442e-a3bf-a24288429beb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:13.325605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f267278f7aef48a4a637847539805c0d', 'request_id': 'req-874ae9e3-1194-442e-a3bf-a24288429beb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:13.337011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] The token cache is being invalidate because project f267278f7aef48a4a637847539805c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:13.337307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:13.337957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-874ae9e3-1194-442e-a3bf-a24288429beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:13.339531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1285/2589] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:12 2022] DELETE /identity/v3/projects/f267278f7aef48a4a637847539805c0d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:13.344509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.344977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.345081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.364010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:13.365275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:13.417950 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:13.426326 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1305/2590] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:13.436165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.436454 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.436739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.455243 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:13.456459 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:13.464423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1286/2591] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:13.471440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.481670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffceabbc-e68a-415b-bc9e-59bb769ae2c9 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.486262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.486953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.487242 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.487522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.488248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:13.488563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.490165 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.502707 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1306/2592] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:13.507106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.517810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be95098-0565-4f53-ba71-055afef26ae6 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.522581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.523259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.523547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.523827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.524519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.526526 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.679403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1287/2593] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:13 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:13.681294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.699530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b4ec8-a379-41c4-b34e-c54a1dcf667c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.761295 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.761980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.762410 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.762728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.763433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.765373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.877769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1307/2594] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:13 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:13.882117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.894073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac0ce8-ec73-4c81-a1d0-2b0a7a336487 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.897708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.898372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.898747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.899538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.902157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.902157 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.912934 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1288/2595] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:13.917681 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.951885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7c35-5059-4081-8725-2d31a5584138 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.951885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.951885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.951885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.952266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.952266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.952266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df509bd-77cd-4a58-a05c-eb3d582c4a59 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:13.952266 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1308/2596] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:13.954487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:13.965595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89829e7-4947-49a9-a98c-edae27c1a2f0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:13.970285 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:13.971180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:13.971467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:13.971747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] PATH_INFO: `/v3/projects/c5f3aaf768974b72bb06839248c27a3e/users/b2ef433a26f544e4b436713bbf23e677/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:13.986960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f3aaf768974b72bb06839248c27a3e, user_id=b2ef433a26f544e4b436713bbf23e677, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:13.989610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:14.178818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1289/2597] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:13 2022] PUT /identity/v3/projects/c5f3aaf768974b72bb06839248c27a3e/users/b2ef433a26f544e4b436713bbf23e677/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:14.189250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.189552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.189837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.217514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] MFA Rules not processed for user `b2ef433a26f544e4b436713bbf23e677`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:14.218663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:14.281492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1309/2598] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:14.286205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:14.354944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e6e1f-d7ae-4485-8e28-3ad48f051afd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:14.359781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:14.360449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.360737 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.361018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.361733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:14.363695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:14.378308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:14.557295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1290/2599] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:14.561616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:14.572861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095effb-3bc6-4350-a9bb-d908dae87696 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:14.623388 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:14.630238 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:14.630980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.631251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.631538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.632241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:14.634838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:14.752092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1310/2600] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:14 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:14.752229 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:14.762473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc86ad-503a-4da3-acf3-fa162d594654 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:14.767337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:14.767988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.768292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.768569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.769316 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:14.771017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:14.782678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1291/2601] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:14.785761 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:14.795812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680fc8-6109-436b-a0f9-fc3e80e69f81 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:14.800980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:14.801330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.801616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.801894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.802680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:14.804285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcee97-6ed9-4765-8b49-7d7ba1ef20a8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:14.815122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1311/2602] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:14.819116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:14.830020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afd0e7-aa10-4784-bd59-296bf7626dbf admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:14.834895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:14.835768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:14.836051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:14.836328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] PATH_INFO: `/v3/projects/5002e7becbc24ea9a9e5be9620514a40/users/71bb247a62f8451cb70df25215609345/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:14.851751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5002e7becbc24ea9a9e5be9620514a40, user_id=71bb247a62f8451cb70df25215609345, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:14.854488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.002502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1292/2603] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:14 2022] PUT /identity/v3/projects/5002e7becbc24ea9a9e5be9620514a40/users/71bb247a62f8451cb70df25215609345/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:15.013603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.013972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.014340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.041948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] MFA Rules not processed for user `71bb247a62f8451cb70df25215609345`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:15.043194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:15.117680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1312/2604] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:15.123172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.187093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ea059-b8c7-4ac6-bd19-adf0f280942a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.191904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.192559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.192846 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.193127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.193834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.195839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.351163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1293/2605] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:15.355929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.366984 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed44b2e-49c9-4e83-84e5-6189d32b81c7 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.424562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.425236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.425523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.425803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.426505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.428468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.544720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1313/2606] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:15 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:15.548909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.559616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48eda9-a82d-40ed-b056-16a2b02beba8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.564195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.564870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.565189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.565474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.566259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.567885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.578582 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1294/2607] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:15.582759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.593536 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254c4d4-ef0a-45fa-a42f-2325d727a9a6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.598292 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.599199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.599487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.599769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] PATH_INFO: `/v3/projects/2e9d84d70b3244729731df6b2aec46a8/users/9df54b2ced7f45928d3a2bbaba85773b/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.614882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9d84d70b3244729731df6b2aec46a8, user_id=9df54b2ced7f45928d3a2bbaba85773b, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.617546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.721302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1314/2608] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:15 2022] PUT /identity/v3/projects/2e9d84d70b3244729731df6b2aec46a8/users/9df54b2ced7f45928d3a2bbaba85773b/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:15.725263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.735881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e56ad9-e299-4f86-92e0-20588547b36c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.791878 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.792550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.794424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.794424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.794424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.795436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8a633-7f4b-49cc-9d21-58ade6999e93 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.808726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1295/2609] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:15.810255 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:15.822343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b8ff0-8fc6-4e5b-b17b-4675647c733c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:15.827338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:15.828354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.828699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.829137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] PATH_INFO: `/v3/projects/2e9d84d70b3244729731df6b2aec46a8/users/9df54b2ced7f45928d3a2bbaba85773b/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.831256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9d84d70b3244729731df6b2aec46a8, user_id=9df54b2ced7f45928d3a2bbaba85773b, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:15.834355 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:15.845900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1315/2610] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:15 2022] PUT /identity/v3/projects/2e9d84d70b3244729731df6b2aec46a8/users/9df54b2ced7f45928d3a2bbaba85773b/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:15.914099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:15.914393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:15.914754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:15.941095 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] MFA Rules not processed for user `9df54b2ced7f45928d3a2bbaba85773b`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:15.942312 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:15.961467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:16.012534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1296/2611] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:16.025426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:16.027142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:16.046472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:16.086615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599f3b-f386-45ab-9d4e-254b25272646 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:16.090387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:16.091055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:16.091340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:16.091619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:16.148409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:16.149868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:16.159003 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1316/2612] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:21.534280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:21.544531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43484b-456b-474e-b41b-51dc6c648d0e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:21.548107 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:21.548849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:21.549138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:21.549466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:21.610457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:21.611921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:21.621812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1297/2613] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:23.275611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:23.284973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce384e3e-ccb6-4a0d-8853-adba52067654 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:23.288724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:23.289362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:23.289651 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:23.289936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:23.300093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:23.301590 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68bffd-8b81-41b5-9096-493756c93ff9 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:23.311423 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1317/2614] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:23.772559 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:23.774161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:23.859360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e0bcb-4559-41c0-a15d-db66e6016f6b service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:23.863986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:23.864887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:23.865175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:23.865467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] PATH_INFO: `/v3/users/acc4fa794e374400b19440d16a0c7521` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:23.866682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc4fa794e374400b19440d16a0c7521)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:23.868657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:24.078382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:24.084081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc4fa794e374400b19440d16a0c7521'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:24.091023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc4fa794e374400b19440d16a0c7521'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:24.091699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc4fa794e374400b19440d16a0c7521'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:24.110028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc4fa794e374400b19440d16a0c7521'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:24.220223 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc4fa794e374400b19440d16a0c7521'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:24.226579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1298/2615] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:23 2022] DELETE /identity/v3/users/acc4fa794e374400b19440d16a0c7521 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:24.231089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:24.236787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:24.236787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:24.259129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:24.260377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:24.319848 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1318/2616] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:24.329711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:24.331195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:24.390938 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e955-6b02-4f01-9f50-9487e0175caf admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:24.394604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:24.396509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:24.396509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:24.396509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:24.405652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:24.407166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:24.417136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1299/2617] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:24.862625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:24.864113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:24.876050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182cdb-7717-44a9-8b3b-8d53777b588b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:24.881033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:24.881889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:24.882173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:24.882552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] PATH_INFO: `/v3/projects/65f8ef529d654842aeaa7c931de23797` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:24.883784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f8ef529d654842aeaa7c931de23797)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:24.885746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:25.022313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:25.025290 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f8ef529d654842aeaa7c931de23797', 'request_id': 'req-f403da30-3516-446c-a0fd-814723b32555'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.128977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f8ef529d654842aeaa7c931de23797', 'request_id': 'req-f403da30-3516-446c-a0fd-814723b32555'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.135046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f8ef529d654842aeaa7c931de23797', 'request_id': 'req-f403da30-3516-446c-a0fd-814723b32555'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.152828 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] The token cache is being invalidate because project 65f8ef529d654842aeaa7c931de23797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:25.153250 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.154900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.157512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1319/2618] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:24 2022] DELETE /identity/v3/projects/65f8ef529d654842aeaa7c931de23797 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:25.162724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:25.173259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d68456-ee87-4db7-bfec-10806379180c service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:25.237745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:25.238612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:25.238946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:25.239224 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] PATH_INFO: `/v3/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:25.240457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a8b0b8b09c49aa9c8d8a306b4b86ca)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:25.242432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:25.455489 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a8b0b8b09c49aa9c8d8a306b4b86ca'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.462547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a8b0b8b09c49aa9c8d8a306b4b86ca'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.463258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a8b0b8b09c49aa9c8d8a306b4b86ca'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.467823 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:25.475529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a8b0b8b09c49aa9c8d8a306b4b86ca'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.580806 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0c1d9aa4-12fd-40a5-895f-1e22e7bf4180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a8b0b8b09c49aa9c8d8a306b4b86ca'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:25.587150 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1300/2619] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:25 2022] DELETE /identity/v3/users/d3a8b0b8b09c49aa9c8d8a306b4b86ca => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:26.044810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.046924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:26.060303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403da30-3516-446c-a0fd-814723b32555 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.117351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.118222 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.118512 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.118839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] PATH_INFO: `/v3/projects/011bf4c710a7476a8cb9797a85e180fc` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.120057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=011bf4c710a7476a8cb9797a85e180fc)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.122010 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.260216 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011bf4c710a7476a8cb9797a85e180fc', 'request_id': 'req-2831b9b1-c490-4928-87dd-d2f30b3b97ee'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:26.346101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011bf4c710a7476a8cb9797a85e180fc', 'request_id': 'req-2831b9b1-c490-4928-87dd-d2f30b3b97ee'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:26.352298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011bf4c710a7476a8cb9797a85e180fc', 'request_id': 'req-2831b9b1-c490-4928-87dd-d2f30b3b97ee'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:26.364375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] The token cache is being invalidate because project 011bf4c710a7476a8cb9797a85e180fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:26.364675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:26.365311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-2831b9b1-c490-4928-87dd-d2f30b3b97ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:26.366893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1320/2620] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:26 2022] DELETE /identity/v3/projects/011bf4c710a7476a8cb9797a85e180fc => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:26.371861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.373498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.373498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.409639 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:26.410920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:26.470194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1301/2621] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:26.478169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.478461 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.478797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.497353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:26.498541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:26.506389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1321/2622] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:26.510671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.520353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae8bf4-5579-4a2c-9757-5e6b488d0914 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.525024 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.525664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.525949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.526227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.526983 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:26.527297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.528873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.540660 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1302/2623] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:26.544964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.555247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6f98e-9bb5-4951-86f2-f658b4a70dac None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.560239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.560883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.561200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.561493 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.562218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.564169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.656540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1322/2624] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:26 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:26.660947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.671600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc502a-2023-4acb-a36b-78c1e370b23d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.730954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.731632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.731917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.732190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.734214 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.736327 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.910944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1303/2625] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:26 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:26.915485 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.925891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54af2-e664-4259-9a14-b6658e45b958 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.930661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.931314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.931602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.931884 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.932610 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.934209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.945754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1323/2626] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:26.951098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.962182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafea79-bfbc-4bba-8c2c-02e796f2d737 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.966808 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:26.967457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:26.968383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:26.968663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:26.969384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:26.971058 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96746a1b-3d10-48c5-b7f2-688f292a9892 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:26.980825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1304/2627] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:26.984902 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:26.994894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351b02d-d831-4364-bdd1-878936fa279f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:26.999616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.000498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.000787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.001067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] PATH_INFO: `/v3/projects/bd868f627f984f17aa5a64cbaa196b95/users/949bfbe7620a4dd1af877f2e09442003/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.016130 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd868f627f984f17aa5a64cbaa196b95, user_id=949bfbe7620a4dd1af877f2e09442003, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.018937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.139080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:27.144314 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1324/2628] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:26 2022] PUT /identity/v3/projects/bd868f627f984f17aa5a64cbaa196b95/users/949bfbe7620a4dd1af877f2e09442003/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:27.154618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.155011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.155857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.182079 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] MFA Rules not processed for user `949bfbe7620a4dd1af877f2e09442003`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:27.183328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:27.201396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:27.250498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1305/2629] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:27.255062 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.331797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9c15-f5fb-4b5b-9bd4-65b61518a4b8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.339660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.340394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.340691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.340975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.341704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.343754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.462497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1325/2630] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:27 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:27.466506 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.477403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e8920-7b0b-4f04-a945-c9c54c678cfe None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.534562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.535346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.535538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.535812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.539010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.541045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.652782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1306/2631] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:27 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:27.656770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.667001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1693833-fb00-4957-9342-6b27d5ad3f1f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.672219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.672872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.673159 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.673438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.674212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.675833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.687247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1326/2632] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:27.691341 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.701408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055b0c-4a0c-43af-996e-b79c5c7678b4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.706112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.707020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.707306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.707589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] PATH_INFO: `/v3/projects/d3a5ce487ed94eb59dd59549518caa52/users/9c8ac3d90b1348fa9a96ad6346dc49d1/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.725533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a5ce487ed94eb59dd59549518caa52, user_id=9c8ac3d90b1348fa9a96ad6346dc49d1, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.726521 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.890121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1307/2633] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:27 2022] PUT /identity/v3/projects/d3a5ce487ed94eb59dd59549518caa52/users/9c8ac3d90b1348fa9a96ad6346dc49d1/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:27.894032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.903932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28317-0964-4fc9-993d-3734ecd23f4c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.962965 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:27.963648 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:27.963933 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:27.964211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:27.964942 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:27.966543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104cdfc-822e-4249-bb89-5568877048d9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:27.977530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1327/2634] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:27.981697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:27.992445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f9ad5-117a-4880-af97-9d4e20d8948b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:27.997218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:28.029283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.029283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.029283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] PATH_INFO: `/v3/projects/d3a5ce487ed94eb59dd59549518caa52/users/9c8ac3d90b1348fa9a96ad6346dc49d1/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.029283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a5ce487ed94eb59dd59549518caa52, user_id=9c8ac3d90b1348fa9a96ad6346dc49d1, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:28.029283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:28.029912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1308/2635] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:27 2022] PUT /identity/v3/projects/d3a5ce487ed94eb59dd59549518caa52/users/9c8ac3d90b1348fa9a96ad6346dc49d1/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:28.040085 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.040391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.040676 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.069021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] MFA Rules not processed for user `9c8ac3d90b1348fa9a96ad6346dc49d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:28.070269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:28.130630 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:28.139440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1328/2636] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:28.150797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:28.153960 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:28.209864 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7d7b5-ae06-4a3f-bf5d-e193e10d04b7 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:28.213594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:28.214295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.214580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.214890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.224563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:28.226021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9338b3-e790-406a-847c-27fb6703de84 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:28.240436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1309/2637] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:28.299773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:28.378252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb43cc2-65d1-41f1-9e6f-9945c6bfdcee None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:28.383644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:28.384559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.384842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.385122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] PATH_INFO: `/v3/users/949bfbe7620a4dd1af877f2e09442003` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.386325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949bfbe7620a4dd1af877f2e09442003)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:28.388376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:28.621325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949bfbe7620a4dd1af877f2e09442003'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:28.628685 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949bfbe7620a4dd1af877f2e09442003'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:28.629441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949bfbe7620a4dd1af877f2e09442003'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:28.644600 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949bfbe7620a4dd1af877f2e09442003'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:28.734507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949bfbe7620a4dd1af877f2e09442003'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:28.741066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1329/2638] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:28 2022] DELETE /identity/v3/users/949bfbe7620a4dd1af877f2e09442003 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:28.745733 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.746225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.746326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.764625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:28.766400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:28.774982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:28.825114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1310/2639] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:28.835735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:28.837169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:28.899074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ab55-f9c9-438b-93ed-ef7b7c61a624 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:28.902673 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:28.903340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:28.903627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:28.903910 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:28.919354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:28.921157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:28.932053 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1330/2640] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:29.402455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:29.404853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:29.416605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ce942-a498-4370-83a3-e947c39a730a None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:29.422748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:29.423632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:29.423921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:29.424196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] PATH_INFO: `/v3/projects/bd868f627f984f17aa5a64cbaa196b95` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:29.425402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd868f627f984f17aa5a64cbaa196b95)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:29.427404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:29.446278 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:29.447834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:29.460827 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ade05-6f96-47ba-b073-09679f4c426b service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:29.466075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:29.467016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:29.467397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:29.468371 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] PATH_INFO: `/v3/users/b2ef433a26f544e4b436713bbf23e677` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:29.469632 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ef433a26f544e4b436713bbf23e677)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:29.471664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:29.487069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:29.652972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ef433a26f544e4b436713bbf23e677'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.654297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd868f627f984f17aa5a64cbaa196b95', 'request_id': 'req-b7e83a03-83bc-4a28-b299-0fef3d4f6714'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.660704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ef433a26f544e4b436713bbf23e677'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.661402 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ef433a26f544e4b436713bbf23e677'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.673931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ef433a26f544e4b436713bbf23e677'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.760794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd868f627f984f17aa5a64cbaa196b95', 'request_id': 'req-b7e83a03-83bc-4a28-b299-0fef3d4f6714'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.767708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd868f627f984f17aa5a64cbaa196b95', 'request_id': 'req-b7e83a03-83bc-4a28-b299-0fef3d4f6714'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.778536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] The token cache is being invalidate because project bd868f627f984f17aa5a64cbaa196b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:29.778857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.779496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.781540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1311/2641] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:29 2022] DELETE /identity/v3/projects/bd868f627f984f17aa5a64cbaa196b95 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:29.785347 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:29.786671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:29.846465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0592b598-ac43-40ae-b062-591f4ca59500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ef433a26f544e4b436713bbf23e677'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:29.850924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e83a03-83bc-4a28-b299-0fef3d4f6714 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:29.853289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1331/2642] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:29 2022] DELETE /identity/v3/users/b2ef433a26f544e4b436713bbf23e677 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:29.855581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:29.856537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:29.856757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:29.857031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] PATH_INFO: `/v3/users/9c8ac3d90b1348fa9a96ad6346dc49d1` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:29.858139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:29.858292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8ac3d90b1348fa9a96ad6346dc49d1)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:29.858486 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:29.858811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:29.860180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:29.879529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:29.880445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:29.940157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1332/2643] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:29.949915 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:29.951517 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:30.016951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c859-c5aa-4a73-adc0-d01be77fc212 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:30.021196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:30.028818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:30.028818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:30.028818 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:30.032591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:30.034193 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:30.043545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1333/2644] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:30.148678 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:30.154531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8ac3d90b1348fa9a96ad6346dc49d1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.162482 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8ac3d90b1348fa9a96ad6346dc49d1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.163764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8ac3d90b1348fa9a96ad6346dc49d1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.175716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8ac3d90b1348fa9a96ad6346dc49d1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.323913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8ac3d90b1348fa9a96ad6346dc49d1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.339155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1312/2645] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:29 2022] DELETE /identity/v3/users/9c8ac3d90b1348fa9a96ad6346dc49d1 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:30.524626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:30.526072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:30.542360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85703f-dd1f-4c14-b4d2-1ed02dca63cc service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:30.583641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:30.602456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:30.603462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:30.603748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:30.604027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] PATH_INFO: `/v3/projects/c5f3aaf768974b72bb06839248c27a3e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:30.605236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f3aaf768974b72bb06839248c27a3e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:30.607334 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:30.770602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f3aaf768974b72bb06839248c27a3e', 'request_id': 'req-cc7de96d-2202-4ec4-addf-be6d7a31647b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.884555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:30.887798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:30.900299 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55977e0-6aad-4e4b-90ba-af60cdea0a03 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:30.917922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f3aaf768974b72bb06839248c27a3e', 'request_id': 'req-cc7de96d-2202-4ec4-addf-be6d7a31647b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.924186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f3aaf768974b72bb06839248c27a3e', 'request_id': 'req-cc7de96d-2202-4ec4-addf-be6d7a31647b'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.936353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] The token cache is being invalidate because project c5f3aaf768974b72bb06839248c27a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:30.937157 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.937824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:30.939504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1334/2646] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:30 2022] DELETE /identity/v3/projects/c5f3aaf768974b72bb06839248c27a3e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:30.943252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:30.944608 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:30.962074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:30.963618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:30.963618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:30.963618 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] PATH_INFO: `/v3/projects/d3a5ce487ed94eb59dd59549518caa52` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:30.964796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a5ce487ed94eb59dd59549518caa52)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:30.966739 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.009720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7de96d-2202-4ec4-addf-be6d7a31647b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.015209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.016117 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.016434 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.016716 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] PATH_INFO: `/v3/users/71bb247a62f8451cb70df25215609345` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.017940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bb247a62f8451cb70df25215609345)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.019971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.070852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a5ce487ed94eb59dd59549518caa52', 'request_id': 'req-0220b9b1-5726-48cf-8fd5-6b34e7d32777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.183629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a5ce487ed94eb59dd59549518caa52', 'request_id': 'req-0220b9b1-5726-48cf-8fd5-6b34e7d32777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.191818 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a5ce487ed94eb59dd59549518caa52', 'request_id': 'req-0220b9b1-5726-48cf-8fd5-6b34e7d32777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.202530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] The token cache is being invalidate because project d3a5ce487ed94eb59dd59549518caa52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:31.202855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.203554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-0220b9b1-5726-48cf-8fd5-6b34e7d32777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.205627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1313/2647] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:30 2022] DELETE /identity/v3/projects/d3a5ce487ed94eb59dd59549518caa52 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:31.210518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd49f-01ad-4a31-9e61-a677f66429fc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.210834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd49f-01ad-4a31-9e61-a677f66429fc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.211114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcd49f-01ad-4a31-9e61-a677f66429fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.229100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3dbcd49f-01ad-4a31-9e61-a677f66429fc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:31.230556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3dbcd49f-01ad-4a31-9e61-a677f66429fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:31.261602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bb247a62f8451cb70df25215609345'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.269261 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bb247a62f8451cb70df25215609345'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.269948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bb247a62f8451cb70df25215609345'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.274694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:31.282172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:31.283123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bb247a62f8451cb70df25215609345'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.292127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1314/2648] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:31.300057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.300329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.300610 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.320710 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:31.322029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:31.331317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1315/2649] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:31.340802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.350332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813e20c-e297-49c8-977a-d9884d3b269d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.355270 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.355949 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.356236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.356514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.357239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:31.357550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.357669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bb247a62f8451cb70df25215609345'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.359747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.365001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1335/2650] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:30 2022] DELETE /identity/v3/users/71bb247a62f8451cb70df25215609345 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:31.371671 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1316/2651] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:06:31.376637 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.378284 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:31.458059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea3cb8-8c75-43ec-b01f-6f5d1682af05 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.462886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.463566 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.463856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.466252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.466252 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.466893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.585253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1336/2652] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:31 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:31.589499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.600570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89890a-63d8-4319-932b-475e9364c4d6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.672774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.674568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.675130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.675643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.676465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.678502 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.808432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1317/2653] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:31 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:31.812923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.822168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.824106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:31.824322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c06a-e8ec-48e9-8722-152a57148d03 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.829213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.830260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.830602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.830927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.831664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.833572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.837946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1101-8856-49e6-9ae1-c52184e40c82 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.843597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.844476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.845049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.845212 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] PATH_INFO: `/v3/projects/5002e7becbc24ea9a9e5be9620514a40` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.846445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5002e7becbc24ea9a9e5be9620514a40)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.848435 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.854359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1337/2654] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:31.856444 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.866940 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefaf96-2df6-4964-97c9-e395c5ab9259 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.871594 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.872262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.872548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.872829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.873554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.875240 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.885386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1338/2655] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:31.889653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:31.900254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7f495-4594-4db2-b32e-c5d9db759c08 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:31.905958 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:31.906560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:31.906880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:31.907160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] PATH_INFO: `/v3/projects/d18e6ced141a422799a53465e8d1942c/users/322315b2f4a940febc61b1d6751f7ff6/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:31.927307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18e6ced141a422799a53465e8d1942c, user_id=322315b2f4a940febc61b1d6751f7ff6, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:31.931272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3028706-dadc-43dd-a402-8afee199247d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:31.939903 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5002e7becbc24ea9a9e5be9620514a40', 'request_id': 'req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.986148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5002e7becbc24ea9a9e5be9620514a40', 'request_id': 'req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:31.992477 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1339/2656] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:31 2022] PUT /identity/v3/projects/d18e6ced141a422799a53465e8d1942c/users/322315b2f4a940febc61b1d6751f7ff6/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:32.001493 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5002e7becbc24ea9a9e5be9620514a40', 'request_id': 'req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.003235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.003731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.004092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.004665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] The token cache is being invalidate because project 5002e7becbc24ea9a9e5be9620514a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:32.004990 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.005664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.007295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1318/2657] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:31 2022] DELETE /identity/v3/projects/5002e7becbc24ea9a9e5be9620514a40 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:32.011196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.012595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.032416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] MFA Rules not processed for user `322315b2f4a940febc61b1d6751f7ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:32.033725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.055958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:32.078098 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf6f00-2fbb-4c7b-ac1c-75b1bf7c0ed1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.100095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1340/2658] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:32.104552 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.105877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.121026 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:32.138320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.139227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.140173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.140367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] PATH_INFO: `/v3/users/9df54b2ced7f45928d3a2bbaba85773b` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.141588 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df54b2ced7f45928d3a2bbaba85773b)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.143849 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.186318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eb6e4-b329-4244-9b4b-2f88733f071d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.191133 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.191789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.192075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.192352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.193042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.195087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.290745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1341/2659] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:32 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:32.294513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.336189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df54b2ced7f45928d3a2bbaba85773b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.344293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df54b2ced7f45928d3a2bbaba85773b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.345052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df54b2ced7f45928d3a2bbaba85773b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.357917 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df54b2ced7f45928d3a2bbaba85773b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.361700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cc57a-b043-463e-82f1-50bfbc53f3ed admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.366445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.367169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.367456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.367735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.368426 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.370343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.434969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df54b2ced7f45928d3a2bbaba85773b'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:32.441848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1319/2660] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:32 2022] DELETE /identity/v3/users/9df54b2ced7f45928d3a2bbaba85773b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:32.476899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1342/2661] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:32 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:32.479728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.481364 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.548162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0e2db-2031-409d-bd57-43b02179f3f5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.552977 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.553674 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.553956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.554232 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.555124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.556841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.568130 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1320/2662] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:32.572304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.583123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2408-ae7c-4390-a8d4-491919f38a0e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.587837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.588705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.589025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.589317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] PATH_INFO: `/v3/projects/8c81fb9566714005a934e4788c5ed5aa/users/54b79066f5bc4bba95fb97b2b867efc8/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.604400 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c81fb9566714005a934e4788c5ed5aa, user_id=54b79066f5bc4bba95fb97b2b867efc8, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.611668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.698248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1343/2663] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:32 2022] PUT /identity/v3/projects/8c81fb9566714005a934e4788c5ed5aa/users/54b79066f5bc4bba95fb97b2b867efc8/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:32.702970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.708320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:32.714533 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148325-04a4-4a86-a5d1-ac67de0dfc69 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.777085 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.777855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.778204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.778574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.779417 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.781032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4e08-60f5-44e2-8f27-8b527bb68d25 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.794722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1321/2664] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:32.796649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.807831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b91a9e-34ab-4840-8642-b7c485095abe admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:32.813660 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:32.814553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.814871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.815188 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] PATH_INFO: `/v3/projects/8c81fb9566714005a934e4788c5ed5aa/users/54b79066f5bc4bba95fb97b2b867efc8/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.817142 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c81fb9566714005a934e4788c5ed5aa, user_id=54b79066f5bc4bba95fb97b2b867efc8, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:32.819932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:32.832200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1344/2665] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:32 2022] PUT /identity/v3/projects/8c81fb9566714005a934e4788c5ed5aa/users/54b79066f5bc4bba95fb97b2b867efc8/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:32.843423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:32.843623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:32.843894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:32.870968 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] MFA Rules not processed for user `54b79066f5bc4bba95fb97b2b867efc8`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:32.872180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.941679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1322/2666] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:32.954022 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.955809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:32.962159 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:32.963688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:33.016108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:33.020054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13c4e-bb74-4d6e-add7-f0a236305482 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:33.024069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:33.024775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:33.025067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:33.025349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:33.035077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:33.036572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:33.040551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4585fa8-e53c-4bd0-81ab-13bfb3963171 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:33.042895 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:33.043874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:33.044173 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:33.044452 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] PATH_INFO: `/v3/projects/2e9d84d70b3244729731df6b2aec46a8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:33.046291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9d84d70b3244729731df6b2aec46a8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:33.047408 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1345/2667] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:33.050043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:33.142673 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:33.159307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9d84d70b3244729731df6b2aec46a8', 'request_id': 'req-2de5452a-83c4-4442-b596-e742ed92fd16'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:33.236632 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9d84d70b3244729731df6b2aec46a8', 'request_id': 'req-2de5452a-83c4-4442-b596-e742ed92fd16'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:33.243817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9d84d70b3244729731df6b2aec46a8', 'request_id': 'req-2de5452a-83c4-4442-b596-e742ed92fd16'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:33.259891 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] The token cache is being invalidate because project 2e9d84d70b3244729731df6b2aec46a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:33.260256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:33.261048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:33.263080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1323/2668] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:32 2022] DELETE /identity/v3/projects/2e9d84d70b3244729731df6b2aec46a8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:34.276924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:34.286626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93db13-c767-4a4f-8a6d-91af282fb70a service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:34.357305 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b79066f5bc4bba95fb97b2b867efc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c81fb9566714005a934e4788c5ed5aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:34.360030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:34.360030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:34.360030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:34.360030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:34.360476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:34.362398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:34.376173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1346/2669] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:34.380429 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:34.382108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:34.396153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5452a-83c4-4442-b596-e742ed92fd16 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:34.400924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b79066f5bc4bba95fb97b2b867efc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c81fb9566714005a934e4788c5ed5aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:34.401665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:34.402028 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:34.402389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:34.403225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:34.405371 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:34.538218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1324/2670] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:34 2022] POST /identity/v3/projects => generated 355 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:34.539379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:34.550306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a873f09-3f4b-4a36-a97b-a77b8c8fd20b tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:34.611797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b79066f5bc4bba95fb97b2b867efc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c81fb9566714005a934e4788c5ed5aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:34.612380 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:34.612670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:34.612954 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:34.613745 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:34.614064 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:34.615703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:34.631192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1347/2671] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:06:34.673317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:34.685856 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7591b64-f351-4254-ad1e-a25892f47eeb tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:34.692044 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b79066f5bc4bba95fb97b2b867efc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c81fb9566714005a934e4788c5ed5aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:34.693017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:34.693408 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:34.694186 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] PATH_INFO: `/v3/projects/b9330eedcaf4431fbc907e383e89d2c8` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:34.695343 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b9330eedcaf4431fbc907e383e89d2c8)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:34.697300 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:34.838164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9330eedcaf4431fbc907e383e89d2c8', 'request_id': 'req-c229351e-8982-483d-8de3-0016d616c154'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:34.960872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9330eedcaf4431fbc907e383e89d2c8', 'request_id': 'req-c229351e-8982-483d-8de3-0016d616c154'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:34.968045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9330eedcaf4431fbc907e383e89d2c8', 'request_id': 'req-c229351e-8982-483d-8de3-0016d616c154'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:34.979010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] The token cache is being invalidate because project b9330eedcaf4431fbc907e383e89d2c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:34.979308 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:34.980018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c229351e-8982-483d-8de3-0016d616c154 tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:34.981594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1325/2672] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:34 2022] DELETE /identity/v3/projects/b9330eedcaf4431fbc907e383e89d2c8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:35.755835 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:35.757636 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:35.825318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef10a0-bd91-4173-9135-60da3b03bf9f tempest-VolumeQuotaClassesTest-686845067 tempest-VolumeQuotaClassesTest-686845067-project-admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:35.850254 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:35.880888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:35.881678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:35.881964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:35.882239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] PATH_INFO: `/v3/users/322315b2f4a940febc61b1d6751f7ff6` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:35.883480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322315b2f4a940febc61b1d6751f7ff6)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:35.885573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:36.068849 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322315b2f4a940febc61b1d6751f7ff6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:36.075721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322315b2f4a940febc61b1d6751f7ff6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:36.076405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322315b2f4a940febc61b1d6751f7ff6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:36.088672 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322315b2f4a940febc61b1d6751f7ff6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:36.163713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322315b2f4a940febc61b1d6751f7ff6'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:36.170257 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1348/2673] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:35 2022] DELETE /identity/v3/users/322315b2f4a940febc61b1d6751f7ff6 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:36.175344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:36.175885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:36.175885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:36.196706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:36.196706 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:36.261081 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1326/2674] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:36.270620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:36.272074 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:36.341934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28df933-bb7f-49f9-9fec-da3fde341da1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:36.346313 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:36.347093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:36.347376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:36.347653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:36.356985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:36.358996 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:36.368612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1349/2675] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:36.859310 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:36.866797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:36.873455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb9a57-d981-4928-8417-990ef7735808 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:36.878361 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:36.879379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:36.879725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:36.880087 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] PATH_INFO: `/v3/projects/d18e6ced141a422799a53465e8d1942c` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:36.881377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18e6ced141a422799a53465e8d1942c)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:36.883445 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:36.895963 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:37.023874 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18e6ced141a422799a53465e8d1942c', 'request_id': 'req-a1256125-b47b-4871-a9a1-608e3cd288cd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.101301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18e6ced141a422799a53465e8d1942c', 'request_id': 'req-a1256125-b47b-4871-a9a1-608e3cd288cd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.109790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18e6ced141a422799a53465e8d1942c', 'request_id': 'req-a1256125-b47b-4871-a9a1-608e3cd288cd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.120950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] The token cache is being invalidate because project d18e6ced141a422799a53465e8d1942c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:37.121240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.122137 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.126872 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1327/2676] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:36 2022] DELETE /identity/v3/projects/d18e6ced141a422799a53465e8d1942c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:37.128377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:37.138541 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc352350-f795-4700-8213-3ec7304c5193 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:37.181811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:37.201058 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:37.204391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:37.204391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:37.204391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] PATH_INFO: `/v3/users/54b79066f5bc4bba95fb97b2b867efc8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:37.204391 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b79066f5bc4bba95fb97b2b867efc8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:37.206576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:37.430219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b79066f5bc4bba95fb97b2b867efc8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.439875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b79066f5bc4bba95fb97b2b867efc8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.440714 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b79066f5bc4bba95fb97b2b867efc8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.456941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b79066f5bc4bba95fb97b2b867efc8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.543478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f886c93d-d3cd-48a0-be75-9f7d3d996cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b79066f5bc4bba95fb97b2b867efc8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:37.549817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1350/2677] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:37 2022] DELETE /identity/v3/users/54b79066f5bc4bba95fb97b2b867efc8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:37.933393 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:37.935248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:37.949321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1256125-b47b-4871-a9a1-608e3cd288cd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.007288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.008141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.008428 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.008705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] PATH_INFO: `/v3/projects/8c81fb9566714005a934e4788c5ed5aa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.009890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c81fb9566714005a934e4788c5ed5aa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.011892 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.198727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c81fb9566714005a934e4788c5ed5aa', 'request_id': 'req-a44fbfc0-f118-495c-9437-e30e82b991e2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:38.380468 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c81fb9566714005a934e4788c5ed5aa', 'request_id': 'req-a44fbfc0-f118-495c-9437-e30e82b991e2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:38.387467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c81fb9566714005a934e4788c5ed5aa', 'request_id': 'req-a44fbfc0-f118-495c-9437-e30e82b991e2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:38.398016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] The token cache is being invalidate because project 8c81fb9566714005a934e4788c5ed5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:38.398306 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:38.398947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-a44fbfc0-f118-495c-9437-e30e82b991e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:38.400571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1328/2678] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:37 2022] DELETE /identity/v3/projects/8c81fb9566714005a934e4788c5ed5aa => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:38.405511 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.405986 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.406136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.434904 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:38.436137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:38.500796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1351/2679] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:38.504998 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.505357 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.505720 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.526804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:38.526804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:38.538174 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1329/2680] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:38.542445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.553086 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511177ce-b7db-4d12-94b6-ed06ab518913 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.557807 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.558465 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.558787 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.559066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.560059 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:38.560367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.563335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.574678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1352/2681] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:06:38.578574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.588182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f8b55-24c6-4836-af82-d9a3409c2af5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.593451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.594114 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.594398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.594709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.595541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.597426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.727571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:38.734714 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1330/2682] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:38 2022] POST /identity/v3/projects => generated 401 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:38.739105 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.749416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ef045-77d5-4a0a-bd1c-967847b5ac32 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.782319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:38.805320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.805988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.806275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.806555 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.807311 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.809270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.912609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1353/2683] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:38 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:38.916749 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.927091 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231aaa2-aec2-4a92-881c-07f0242b24b9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.932042 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.932725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.933051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.934096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.935247 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.937060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.948116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1331/2684] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:38.953122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.963411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16f9f-e4bb-4650-a008-58a122402ed1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:38.968861 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:38.972911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:38.972911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:38.972911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:38.972911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:38.972911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d577-99b8-47f0-9167-99e820d0a7fa admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:38.982724 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1354/2685] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:38.986839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:38.997211 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efaee75-e254-42a3-b8e8-0b6cf856bdc4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.001850 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.002786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.003075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.003355 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] PATH_INFO: `/v3/projects/6e6d434e69984e329494c1ba5f238c69/users/0d6c6dc923834bd6827a16e753adb465/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.018771 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6d434e69984e329494c1ba5f238c69, user_id=0d6c6dc923834bd6827a16e753adb465, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.024602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.138185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1332/2686] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:38 2022] PUT /identity/v3/projects/6e6d434e69984e329494c1ba5f238c69/users/0d6c6dc923834bd6827a16e753adb465/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:39.149275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.149689 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.149899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.177061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] MFA Rules not processed for user `0d6c6dc923834bd6827a16e753adb465`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:39.178353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:39.243182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1355/2687] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:39.247768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.311416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5da091-9509-4f96-a1a4-ad54585255f4 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.318708 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.319367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.319650 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.319928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.320629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.322567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.451663 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1333/2688] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:39 2022] POST /identity/v3/projects => generated 399 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:39.456385 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.467416 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af7d7e-18f4-4326-9044-10faf1ac70d4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.524182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.524781 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.525068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.525351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.526036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.528013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.609618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1356/2689] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:39 2022] POST /identity/v3/users => generated 306 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:39.613568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.623975 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a696c-7144-46b0-81bd-c46b4fff542f admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.629317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.629969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.630344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.630626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.631431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.633023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.644115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1334/2690] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:39.652498 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.662691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab370f52-c78a-4bef-9773-2f3f33767b10 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.667354 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.668221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.668509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.668789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] PATH_INFO: `/v3/projects/9413744afd684145be0f118e6876d553/users/0107ca237fef420584d8dac041a6612e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.684100 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9413744afd684145be0f118e6876d553, user_id=0107ca237fef420584d8dac041a6612e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.687718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.755550 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1357/2691] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:39 2022] PUT /identity/v3/projects/9413744afd684145be0f118e6876d553/users/0107ca237fef420584d8dac041a6612e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:39.759630 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.769699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2c85b-c87d-4af3-a307-60c643a08c13 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.828046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.828636 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.828920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.829196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.829910 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.831512 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbaf76-ab7b-4093-afba-4f86dfea0fb8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.841580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1335/2692] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:39.845788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:39.856277 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf51f2f-077a-4234-b955-a450a22ebd24 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:39.861384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:39.862247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.862532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.862879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] PATH_INFO: `/v3/projects/9413744afd684145be0f118e6876d553/users/0107ca237fef420584d8dac041a6612e/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.865104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9413744afd684145be0f118e6876d553, user_id=0107ca237fef420584d8dac041a6612e, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:39.867773 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:39.873337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:39.879316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1358/2693] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:39 2022] PUT /identity/v3/projects/9413744afd684145be0f118e6876d553/users/0107ca237fef420584d8dac041a6612e/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:39.892764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:39.893051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:39.893332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:39.923258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] MFA Rules not processed for user `0107ca237fef420584d8dac041a6612e`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:39.924510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:39.955723 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:40.014176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1336/2694] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:40.024109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:40.026174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:40.087757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeaf54-0c73-482e-b5a0-9d1170f8dbe0 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:40.091446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:40.092099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:40.092386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:40.092668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:40.102794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:40.104319 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:40.114547 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1359/2695] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:40.374450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:40.375890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:40.388658 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5a74-7a53-4ea1-887e-339facb4cebe None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:40.393009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:40.393662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:40.394022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:40.394302 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:40.451579 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:40.453558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:40.462988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1337/2696] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:41.432509 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:41.433260 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:41.491155 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c069d7-445e-4832-bb30-4ad38135d34f service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:41.494939 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:41.495577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:41.495857 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:41.496132 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:41.497966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:41.561491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:41.563014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:41.572759 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1360/2697] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:45.161719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:45.163283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:45.183556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03affd-f7c9-48f8-b932-5953457aadcb service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:45.189101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:45.189989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:45.190279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:45.190554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] PATH_INFO: `/v3/users/cd25c0ad8e1c4fa08eca9b6bddeca55d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:45.191967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd25c0ad8e1c4fa08eca9b6bddeca55d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:45.193967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:45.421899 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd25c0ad8e1c4fa08eca9b6bddeca55d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.429477 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd25c0ad8e1c4fa08eca9b6bddeca55d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.430244 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd25c0ad8e1c4fa08eca9b6bddeca55d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.443416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd25c0ad8e1c4fa08eca9b6bddeca55d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.520531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd25c0ad8e1c4fa08eca9b6bddeca55d'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.527112 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1338/2698] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:45 2022] DELETE /identity/v3/users/cd25c0ad8e1c4fa08eca9b6bddeca55d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:45.649927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:45.660794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3c5f0-67c9-434a-9ac6-20f32e94d2dd service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:45.708696 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=100 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:45.719208 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:45.720095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:45.721534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:45.721534 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] PATH_INFO: `/v3/users/0d6c6dc923834bd6827a16e753adb465` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:45.721797 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6c6dc923834bd6827a16e753adb465)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:45.723985 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:45.816269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:45.817709 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:45.836128 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e22ca9-0916-4638-a2e2-8605f1163330 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:45.841793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:45.842617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:45.843601 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:45.843855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] PATH_INFO: `/v3/projects/c86222a9c32a4f77878dc82c40e38ff1` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:45.845043 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86222a9c32a4f77878dc82c40e38ff1)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:45.847053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:45.903245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6c6dc923834bd6827a16e753adb465'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.910883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6c6dc923834bd6827a16e753adb465'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.911588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6c6dc923834bd6827a16e753adb465'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.923589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6c6dc923834bd6827a16e753adb465'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:45.987470 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:46.004019 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86222a9c32a4f77878dc82c40e38ff1', 'request_id': 'req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.025900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-f6a76d1d-7d04-432d-948a-43334c902d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6c6dc923834bd6827a16e753adb465'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.032504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1361/2699] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:45 2022] DELETE /identity/v3/users/0d6c6dc923834bd6827a16e753adb465 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:46.037253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.037572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.037894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.056783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:46.057994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.109391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86222a9c32a4f77878dc82c40e38ff1', 'request_id': 'req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.116421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86222a9c32a4f77878dc82c40e38ff1', 'request_id': 'req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.116577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1362/2700] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:46.126784 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.127906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] The token cache is being invalidate because project c86222a9c32a4f77878dc82c40e38ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:46.128122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.128241 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.128853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a4027a5-5fdf-47f3-9023-fb2e6f46b057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.130442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1339/2701] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:45 2022] DELETE /identity/v3/projects/c86222a9c32a4f77878dc82c40e38ff1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:46.135105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f091ea-2ff7-4a13-8c3b-b99784aa3688 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.135397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f091ea-2ff7-4a13-8c3b-b99784aa3688 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.135679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f091ea-2ff7-4a13-8c3b-b99784aa3688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.158668 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-e1f091ea-2ff7-4a13-8c3b-b99784aa3688 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:46.159897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-e1f091ea-2ff7-4a13-8c3b-b99784aa3688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.168240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1340/2702] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:46.176354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.176646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.176931 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.188000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47dde0-6240-4c0b-8d37-8143f2f7a3b4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.191920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.192695 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.192982 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.193258 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.195194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.195629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:46.196809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.205426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1341/2703] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:46.208333 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.209570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.209845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.218886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3e548-fcbc-457b-9b25-58a54f7b5b23 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.219770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1363/2704] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:46.223751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.224403 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.224690 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.224966 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.225691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:46.226002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.227631 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.239789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1342/2705] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:06:46.243870 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.254575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca979-730a-4dc2-b6ee-4a0d582090c7 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.259885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.260537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.261108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.261108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.261868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.263809 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.274620 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:46.408325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1364/2706] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:46.409519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.419598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fd4f4-47a7-4422-8899-0392598552b5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.487682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.488359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.488647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.488972 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.489656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.491684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.580626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1343/2707] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/users => generated 282 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:46.584556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.595218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d8ed-07af-451f-919d-3db5b77ccbb1 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.599826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.600491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.600777 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.601057 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.601788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.603524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.614468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1365/2708] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:46.618451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.629226 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b379f-e843-4f17-9684-4d6be7db549d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.633896 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.634537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.634913 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.635193 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.635918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.637494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.651138 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1344/2709] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:46.651331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.652134 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.652644 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.666088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9299ed5-e890-4dad-aea2-1b2ca6766af0 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.667081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b63d1-050f-43a1-8f86-d461c70ecc84 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:46.668907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.669801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.670084 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.670406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] PATH_INFO: `/v3/projects/a6a0418efcbb433d957b7639d327b896/users/8bbc4e427000488cb197d74222d7439c/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.673583 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:46.674521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.674909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.675265 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] PATH_INFO: `/v3/projects/6e6d434e69984e329494c1ba5f238c69` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.676746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6d434e69984e329494c1ba5f238c69)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.679004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.688329 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a0418efcbb433d957b7639d327b896, user_id=8bbc4e427000488cb197d74222d7439c, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:46.691181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e4164-716c-41bf-95e9-f7a2569457a6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:46.830033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1345/2710] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:46 2022] PUT /identity/v3/projects/a6a0418efcbb433d957b7639d327b896/users/8bbc4e427000488cb197d74222d7439c/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:46.840198 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:46.840519 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:46.840801 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:46.862794 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6d434e69984e329494c1ba5f238c69', 'request_id': 'req-24dfdf75-eb1a-411c-940f-7a1fc3791e60'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.870013 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] MFA Rules not processed for user `8bbc4e427000488cb197d74222d7439c`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:46.871258 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:46.900718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:46.935249 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1346/2711] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:46.939745 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.943331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6d434e69984e329494c1ba5f238c69', 'request_id': 'req-24dfdf75-eb1a-411c-940f-7a1fc3791e60'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.949514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6d434e69984e329494c1ba5f238c69', 'request_id': 'req-24dfdf75-eb1a-411c-940f-7a1fc3791e60'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.960888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] The token cache is being invalidate because project 6e6d434e69984e329494c1ba5f238c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:46.961177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.961922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:46.963520 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1366/2712] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:46 2022] DELETE /identity/v3/projects/6e6d434e69984e329494c1ba5f238c69 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:46.967548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:46.969523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.002947 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f1f49-017f-4c8d-91c3-464338a7cba0 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.007833 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.008492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.008786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.009203 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.009944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.011928 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.033455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdf75-eb1a-411c-940f-7a1fc3791e60 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.038196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.039108 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.039399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.039677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] PATH_INFO: `/v3/users/0107ca237fef420584d8dac041a6612e` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.040877 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0107ca237fef420584d8dac041a6612e)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.042872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.144785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1347/2713] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:46 2022] POST /identity/v3/projects => generated 363 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:47.148717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.150350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.213861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24c3d3-8037-4581-8b7e-bf5af4965fdd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.218580 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.219307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.219591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.219866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.220547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.222451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.336049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0107ca237fef420584d8dac041a6612e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:47.339478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1348/2714] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:47 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:47.341970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.346323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0107ca237fef420584d8dac041a6612e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:47.347087 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0107ca237fef420584d8dac041a6612e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:47.352124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d0ea-481c-4d6c-bc74-5c6addd67e41 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.354617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:47.356887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.357536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.357820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.358093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.359384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.360539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0107ca237fef420584d8dac041a6612e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:47.361001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.371340 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1349/2715] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:47.376006 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.385711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00144ac8-5215-4ee9-acab-dcc59e98988a admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.390277 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.391163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.391449 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.391728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] PATH_INFO: `/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15/users/b45788509ff64e00a39a1cb4657b17e9/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.406656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3560405f4f1a4a0e8b33f1d639a8cc15, user_id=b45788509ff64e00a39a1cb4657b17e9, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.409299 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.440457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0107ca237fef420584d8dac041a6612e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:47.447015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1367/2716] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:46 2022] DELETE /identity/v3/users/0107ca237fef420584d8dac041a6612e => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:47.487037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1350/2717] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:47 2022] PUT /identity/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15/users/b45788509ff64e00a39a1cb4657b17e9/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:47.491147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.492548 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.556580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e36c3a-b130-4327-8e90-992ee33acbcb admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.612433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.613113 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.613401 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.613680 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.614413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.616063 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8064e14-fe3e-4f69-a3c6-30eb6b0f0d87 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.628842 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1368/2718] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:47.630154 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.640909 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e6d1-c970-4d9e-a090-7bbb2e2c3e57 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.645653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.646536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.646857 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.647141 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] PATH_INFO: `/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15/users/b45788509ff64e00a39a1cb4657b17e9/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.649117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3560405f4f1a4a0e8b33f1d639a8cc15, user_id=b45788509ff64e00a39a1cb4657b17e9, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.651939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.659973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:47.666280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1351/2719] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:47 2022] PUT /identity/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15/users/b45788509ff64e00a39a1cb4657b17e9/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:47.679678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.680197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.680336 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.707489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] MFA Rules not processed for user `b45788509ff64e00a39a1cb4657b17e9`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:47.708767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.782204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1369/2720] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:47.793740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.805160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.854587 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe23c-738e-4bde-b872-1a2ed030c228 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.858574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.859413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.859703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.859984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.862437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:47.864211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:47.872346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.874074 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:47.885283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1352/2721] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:47.892102 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:47.945304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85de5b-786a-46ca-8e63-38a32ee624e0 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:47.949882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:47.950811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:47.951125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:47.956395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] PATH_INFO: `/v3/projects/9413744afd684145be0f118e6876d553` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:47.956395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9413744afd684145be0f118e6876d553)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:47.956395 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.077270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9413744afd684145be0f118e6876d553', 'request_id': 'req-066c488b-304e-42e1-8d23-d6f1eec4b82e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:48.107509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.116455 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630088b5-70c1-4542-a38b-f8c889a896e7 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.158710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9413744afd684145be0f118e6876d553', 'request_id': 'req-066c488b-304e-42e1-8d23-d6f1eec4b82e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:48.165468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9413744afd684145be0f118e6876d553', 'request_id': 'req-066c488b-304e-42e1-8d23-d6f1eec4b82e'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:48.166425 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.167115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.167404 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.167689 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.169503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:48.176750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] The token cache is being invalidate because project 9413744afd684145be0f118e6876d553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:48.177027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:48.177677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-066c488b-304e-42e1-8d23-d6f1eec4b82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:48.179294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1370/2722] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:47 2022] DELETE /identity/v3/projects/9413744afd684145be0f118e6876d553 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:48.184109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.184686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.185066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.205814 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:48.207149 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:48.229802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.231528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e1a42-199d-4942-b264-6c972ee20a4a service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.240594 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1353/2723] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:48.268139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1371/2724] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:48.275952 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.276313 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.276597 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.294811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:48.296061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:48.305164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1354/2725] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:48.309663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.320307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce62b-a50e-45aa-9b08-8faa6585355b None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.325009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.325669 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.325957 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.326237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.327013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:48.327323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.329553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.341574 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1372/2726] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:48.346088 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.355481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9790f9-2121-4a20-b351-e00de95d0bcc None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.360204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.360873 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.361152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.364554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.364554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.364554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.554346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1355/2727] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:48 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:48.558491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.569964 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb359-9090-460f-a582-33c071eeb4db admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.595871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:48.626479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.627244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.627529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.627847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.628529 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.630437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.753012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1373/2728] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:48 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:48.757446 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.770604 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41651ba-2fe5-435e-b057-4470829e4762 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.778317 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.779039 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.779383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.779612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.781195 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.782055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.805662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1356/2729] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:48.806890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.818005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147e82-3e91-47a8-92aa-d19bccdc20ca admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.823505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.824170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.824457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.824736 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.825469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.827107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fc3e3-aa12-42c4-9618-820dfbea1c9f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.837279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1374/2730] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:48.841561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:48.852079 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18767d2e-1f3b-44c7-80e1-1dbe113809c5 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:48.857117 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:48.858009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.858297 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.858606 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] PATH_INFO: `/v3/projects/7c34ba9e9d8440a7b01d55676e598955/users/dc69aa767cac432d92de44530e671777/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:48.873754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:48.876068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c34ba9e9d8440a7b01d55676e598955, user_id=dc69aa767cac432d92de44530e671777, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:48.878921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:48.975661 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1357/2731] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:48 2022] PUT /identity/v3/projects/7c34ba9e9d8440a7b01d55676e598955/users/dc69aa767cac432d92de44530e671777/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:48.987432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:48.987732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:48.988017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.013993 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] MFA Rules not processed for user `dc69aa767cac432d92de44530e671777`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:49.015262 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:49.081042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1375/2732] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:49.085679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.147458 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592728d-61d8-447e-8f5c-6fb25d6c82d1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.152325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.152976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.153261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.153541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.154250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.156178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.274017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1358/2733] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:49 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:49.278283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.289769 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb88da-4461-440a-99d1-8ac3e3f9be6d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.345819 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.346490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.347197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.347197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.347770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.349698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.443889 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1376/2734] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:49 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:49.448026 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.459080 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a9e9a-7d8a-4ba4-925b-8b890fb2db34 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.464029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.464727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.465009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.465402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.466133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.467788 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.480473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1359/2735] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:49.489894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.499928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ddd53-3544-40bc-be1e-ffe60d41613a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.504453 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.505361 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.505643 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.505918 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] PATH_INFO: `/v3/projects/d6f3cd06ef294e67a4d4364abab53e49/users/4dac257af006475e9a47ede73c5f3822/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.521414 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f3cd06ef294e67a4d4364abab53e49, user_id=4dac257af006475e9a47ede73c5f3822, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.524189 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.649316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1377/2736] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:49 2022] PUT /identity/v3/projects/d6f3cd06ef294e67a4d4364abab53e49/users/4dac257af006475e9a47ede73c5f3822/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:49.653172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.664946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272edde-bae6-46e2-9d10-bc78702f44e8 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.721108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.721768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.722051 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.722328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.723082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.724669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622fd57-8ee8-4e2f-b0c9-86f3e93b0339 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.734648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1360/2737] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:49.738715 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.744109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:49.749304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a2166-5865-4c96-a46a-a5dacc22de77 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.754571 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.762153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.762153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.762153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] PATH_INFO: `/v3/projects/d6f3cd06ef294e67a4d4364abab53e49/users/4dac257af006475e9a47ede73c5f3822/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.762153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f3cd06ef294e67a4d4364abab53e49, user_id=4dac257af006475e9a47ede73c5f3822, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.762153 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:49.772038 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1378/2738] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:49 2022] PUT /identity/v3/projects/d6f3cd06ef294e67a4d4364abab53e49/users/4dac257af006475e9a47ede73c5f3822/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:49.782192 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.782487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.782805 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.810295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] MFA Rules not processed for user `4dac257af006475e9a47ede73c5f3822`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:49.811558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:49.864625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:49.885740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1361/2739] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:49.908591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:49.910310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:49.969012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aec0e-36f9-484f-ac64-ee68537eceda admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:49.972730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:49.973421 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:49.973708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:49.973992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:49.983329 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:49.984789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:50.003198 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1379/2740] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:50.091291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:50.101194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221efcfd-aa71-4c63-b3cd-5a28606ef193 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:50.104885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:50.105518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:50.105802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:50.106077 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:50.163073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:50.164571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49105140-fead-404c-aaf2-28149cc6e967 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:50.174855 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1362/2741] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:52.065562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:52.067072 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:52.132495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239c0-39f7-4701-bac9-e4de3642588d service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:52.137212 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:52.138077 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:52.138362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:52.138735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] PATH_INFO: `/v3/users/8bbc4e427000488cb197d74222d7439c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:52.139956 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbc4e427000488cb197d74222d7439c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:52.141963 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:52.374433 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbc4e427000488cb197d74222d7439c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:52.381701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbc4e427000488cb197d74222d7439c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:52.382398 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbc4e427000488cb197d74222d7439c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:52.394704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbc4e427000488cb197d74222d7439c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:52.480274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbc4e427000488cb197d74222d7439c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:52.487435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1380/2742] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:52 2022] DELETE /identity/v3/users/8bbc4e427000488cb197d74222d7439c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:52.492296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:52.492881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:52.492881 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:52.510893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:52.512188 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:52.571792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1363/2743] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:52.581585 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:52.583021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:52.596650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:52.639771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d804d3f-0582-411b-8a07-06dcafbdfd6f admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:52.644081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:52.647735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:52.647735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:52.647735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:52.654263 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:52.655748 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:52.666107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1381/2744] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:53.070664 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:53.072129 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:53.085238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982434c-d289-4ada-9ada-8f5e00d82f92 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:53.089798 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:53.090688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:53.090982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:53.091261 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] PATH_INFO: `/v3/projects/a6a0418efcbb433d957b7639d327b896` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:53.092451 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a0418efcbb433d957b7639d327b896)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:53.094376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:53.200993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a0418efcbb433d957b7639d327b896', 'request_id': 'req-86954ca1-091b-4da2-b879-f02ca7ac6cbb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.265933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a0418efcbb433d957b7639d327b896', 'request_id': 'req-86954ca1-091b-4da2-b879-f02ca7ac6cbb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.273121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a0418efcbb433d957b7639d327b896', 'request_id': 'req-86954ca1-091b-4da2-b879-f02ca7ac6cbb'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.284669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] The token cache is being invalidate because project a6a0418efcbb433d957b7639d327b896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:53.284959 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.285602 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.287189 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1364/2745] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:53 2022] DELETE /identity/v3/projects/a6a0418efcbb433d957b7639d327b896 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:53.291289 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:53.301833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ee30-c568-46a1-9769-ae9a33aa9a29 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:53.324359 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:53.325976 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:53.337625 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:53.363140 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:53.363808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:53.364096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:53.364437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] PATH_INFO: `/v3/users/b45788509ff64e00a39a1cb4657b17e9` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:53.365747 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45788509ff64e00a39a1cb4657b17e9)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:53.367858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:53.393719 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86954ca1-091b-4da2-b879-f02ca7ac6cbb admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:53.398591 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:53.399508 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:53.399792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:53.400075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] PATH_INFO: `/v3/users/dc69aa767cac432d92de44530e671777` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:53.401325 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc69aa767cac432d92de44530e671777)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:53.403304 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:53.597618 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45788509ff64e00a39a1cb4657b17e9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.604459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45788509ff64e00a39a1cb4657b17e9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.605275 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45788509ff64e00a39a1cb4657b17e9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.617668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45788509ff64e00a39a1cb4657b17e9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.632859 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc69aa767cac432d92de44530e671777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.640030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc69aa767cac432d92de44530e671777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.640702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc69aa767cac432d92de44530e671777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.653021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc69aa767cac432d92de44530e671777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.743885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-0844d9be-7ac6-42dd-9d62-0a3d94c16bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45788509ff64e00a39a1cb4657b17e9'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.750523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1382/2746] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:53 2022] DELETE /identity/v3/users/b45788509ff64e00a39a1cb4657b17e9 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:53.820407 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc69aa767cac432d92de44530e671777'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:53.826957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1365/2747] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:53 2022] DELETE /identity/v3/users/dc69aa767cac432d92de44530e671777 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:53.831514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:53.831829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:53.832101 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:53.850516 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:53.851802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:53.882578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:53.912592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1383/2748] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:53.922691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:53.924702 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:53.987848 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f42c5-3844-4fa0-9ed5-bbcb6efb7095 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:53.991589 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:53.992240 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:53.992528 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:53.992811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.002757 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.004696 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:54.014391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1366/2749] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:54.361817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.363273 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:54.376304 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25459f03-4722-464f-81dc-a44a91534cbe None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:54.381091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:54.381896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.382180 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.383016 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] PATH_INFO: `/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.384169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3560405f4f1a4a0e8b33f1d639a8cc15)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.386274 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:54.558241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3560405f4f1a4a0e8b33f1d639a8cc15', 'request_id': 'req-733dc418-43f8-4f82-a70e-4ec7686f2b4a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.584943 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.585376 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:54.599825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079542be-df7b-48d8-b98e-d5e289eb85a8 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:54.630366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:54.643513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3560405f4f1a4a0e8b33f1d639a8cc15', 'request_id': 'req-733dc418-43f8-4f82-a70e-4ec7686f2b4a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.650019 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3560405f4f1a4a0e8b33f1d639a8cc15', 'request_id': 'req-733dc418-43f8-4f82-a70e-4ec7686f2b4a'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.662992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] The token cache is being invalidate because project 3560405f4f1a4a0e8b33f1d639a8cc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:54.663317 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.663440 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:54.664044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-733dc418-43f8-4f82-a70e-4ec7686f2b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.664253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.664549 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.664810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] PATH_INFO: `/v3/projects/7c34ba9e9d8440a7b01d55676e598955` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.665871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1384/2750] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:54 2022] DELETE /identity/v3/projects/3560405f4f1a4a0e8b33f1d639a8cc15 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:54.666030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c34ba9e9d8440a7b01d55676e598955)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.668379 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:54.670539 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e94e-c0a7-40a7-8f45-a64c3b3841d0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.670941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e94e-c0a7-40a7-8f45-a64c3b3841d0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.671228 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e94e-c0a7-40a7-8f45-a64c3b3841d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.690650 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3f51e94e-c0a7-40a7-8f45-a64c3b3841d0 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:54.691936 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3f51e94e-c0a7-40a7-8f45-a64c3b3841d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:54.700106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1385/2751] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:54.710320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.710607 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.710924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.729419 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:54.730803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:54.738810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1386/2752] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:54.744137 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.752994 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b161eea-f3de-422c-a64d-b8e29798c74c None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:54.757725 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:54.758344 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.758626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.758974 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.759691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:54.760001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.761598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:54.773791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1387/2753] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:06:54.777921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.787900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f17a4a-4982-4d18-9f35-70ea3a822b9e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:54.792603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:54.793207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.793491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.793782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.794545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.796432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:54.815235 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c34ba9e9d8440a7b01d55676e598955', 'request_id': 'req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.885187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1388/2754] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:54 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:54.889246 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.921263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c34ba9e9d8440a7b01d55676e598955', 'request_id': 'req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.928705 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c34ba9e9d8440a7b01d55676e598955', 'request_id': 'req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.940360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] The token cache is being invalidate because project 7c34ba9e9d8440a7b01d55676e598955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:54.940646 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.941286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:54.942908 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1367/2755] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:54 2022] DELETE /identity/v3/projects/7c34ba9e9d8440a7b01d55676e598955 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:54.946841 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:54.948647 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:54.953322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0491adb-06d5-4455-9782-204887683f28 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:54.959061 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:54.959992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:54.960298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:54.960578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:54.961269 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:54.963227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.013643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfce7c5-ac15-4a60-8e9c-cd50a831f158 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.018925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.019795 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.020078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.020354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] PATH_INFO: `/v3/users/4dac257af006475e9a47ede73c5f3822` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.021565 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dac257af006475e9a47ede73c5f3822)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.023637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.084483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:55.084483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1389/2756] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:54 2022] POST /identity/v3/users => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:55.086211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.087908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:55.100023 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f5184-ea69-479b-bb72-36bf913ab8e7 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.104578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.105211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.105494 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.105771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.106479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.108182 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.118528 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1390/2757] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:55.122603 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.132042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea96b14-e6f4-4597-8a78-e7e211ed615a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.136623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.137237 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.141808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.141808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.141808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.141808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.150576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1391/2758] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:55.154456 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.164251 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7900e-8d15-427e-ba37-b294ed915ae9 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.168906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.170506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.170506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.170506 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] PATH_INFO: `/v3/projects/8f84476142c9430487a52d95cf1d49ab/users/7fa48e71823b4cdc91989e3c33db9d76/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.186495 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f84476142c9430487a52d95cf1d49ab, user_id=7fa48e71823b4cdc91989e3c33db9d76, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.189173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effabeeb-fe3f-4257-ae1f-8ef694c5fdba admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.302075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dac257af006475e9a47ede73c5f3822'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:55.309096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dac257af006475e9a47ede73c5f3822'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:55.310061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dac257af006475e9a47ede73c5f3822'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:55.322490 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dac257af006475e9a47ede73c5f3822'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:55.353375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1392/2759] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:55 2022] PUT /identity/v3/projects/8f84476142c9430487a52d95cf1d49ab/users/7fa48e71823b4cdc91989e3c33db9d76/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:55.364577 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.365150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.365150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.390294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] MFA Rules not processed for user `7fa48e71823b4cdc91989e3c33db9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:55.391617 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:55.449035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dac257af006475e9a47ede73c5f3822'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:55.454396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1393/2760] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:55.456835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1368/2761] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:06:54 2022] DELETE /identity/v3/users/4dac257af006475e9a47ede73c5f3822 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:55.460757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.523708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b14ed-6d89-4b80-b8a0-5cd4f79c4b78 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.580338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.581024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.581340 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.581649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.582360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.584393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.665615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1394/2762] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:55 2022] POST /identity/v3/projects => generated 407 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:55.668234 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.670142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:55.686115 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850582-0117-4cc6-9c4a-8f3ed48b5eb3 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.744498 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.745292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.745474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.745740 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.746436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.748398 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.858532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1369/2763] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:55 2022] POST /identity/v3/users => generated 310 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:55.862665 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.873011 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ceffb-9ee9-4249-a6b8-934b7d73c4ea admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.878161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.878841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.879138 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.879562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.880184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.881321 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:55.881786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.889221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.891191 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:55.893740 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1395/2764] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:55.897833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:55.904473 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc2f8-5ba8-4f03-a23a-8e873ff76a61 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.908664 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d64a0e-3206-448a-9861-675d100335a6 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:55.910349 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.911286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.911572 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.911853 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] PATH_INFO: `/v3/projects/d6f3cd06ef294e67a4d4364abab53e49` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.913052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f3cd06ef294e67a4d4364abab53e49)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.913729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:55.914621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:55.914948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:55.915105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:55.915231 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] PATH_INFO: `/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832/users/def65052cb774692af13018ce62be5fa/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:55.931538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8e3df8c1034013a7a8ec7fcb98f832, user_id=def65052cb774692af13018ce62be5fa, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:55.934299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.041363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1396/2765] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:55 2022] PUT /identity/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832/users/def65052cb774692af13018ce62be5fa/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:06:56.045177 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.078619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f3cd06ef294e67a4d4364abab53e49', 'request_id': 'req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:56.106804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f3823-6ff8-4cd8-88d7-2aafc9872407 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.160934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.161578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.161875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.162148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.162908 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.164497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.175115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1397/2766] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:56.178831 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.188405 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f8d66-6595-4bef-8d38-b9ee48ad750a admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.191948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f3cd06ef294e67a4d4364abab53e49', 'request_id': 'req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:56.194000 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.194934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.195223 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.195507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] PATH_INFO: `/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832/users/def65052cb774692af13018ce62be5fa/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.197537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8e3df8c1034013a7a8ec7fcb98f832, user_id=def65052cb774692af13018ce62be5fa, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.200166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f3cd06ef294e67a4d4364abab53e49', 'request_id': 'req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:56.200335 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba5076-f768-4838-a04d-9bfa14477516 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.210810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1398/2767] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:56 2022] PUT /identity/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832/users/def65052cb774692af13018ce62be5fa/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:56.211252 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] The token cache is being invalidate because project d6f3cd06ef294e67a4d4364abab53e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:06:56.211542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:56.212205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-85ea72fd-1b0e-4474-96ab-4b24c8f6d21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:06:56.213754 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1370/2768] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:06:55 2022] DELETE /identity/v3/projects/d6f3cd06ef294e67a4d4364abab53e49 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:56.218771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472f6a-ddd2-4d81-8e49-fc8ad91ab9ba None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.219065 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472f6a-ddd2-4d81-8e49-fc8ad91ab9ba None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.219351 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19472f6a-ddd2-4d81-8e49-fc8ad91ab9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.221629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.221993 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.222293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.237983 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-19472f6a-ddd2-4d81-8e49-fc8ad91ab9ba None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:56.239207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-19472f6a-ddd2-4d81-8e49-fc8ad91ab9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.250030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] MFA Rules not processed for user `def65052cb774692af13018ce62be5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:56.250377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:56.251278 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.301863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1399/2769] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:56.310291 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.310579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.310981 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.327432 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1371/2770] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:56.336032 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:56.337921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.344701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.345692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.348711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1400/2771] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:56.350245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:56.353560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.368166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc816f25-a703-4eca-8b32-3d9603740515 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.374092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.374943 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.375294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.375721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.377455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:06:56.378116 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.380538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.383519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:56.394451 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1401/2772] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:06:56.398872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.409376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9dd43-5f41-47b1-b9a7-3a640b132c9e admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.416241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.416241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.416241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.416241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.416675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0914c-0b5c-4715-b08a-8e8664dc46cb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.416782 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.418030 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.419835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.420486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.420774 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.421055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.431045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.432494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.442712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1372/2773] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:56.494312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1402/2774] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:06:56.498390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.508765 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a07615-db90-49a4-837f-2513155350a9 service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.566944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.567614 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.567894 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.568171 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.568868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.570826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.635375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1373/2775] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:06:56.639825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.650430 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7bf60-6f72-40fa-af90-5186083011f4 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.656729 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.657424 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.657701 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.658519 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.659270 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.660893 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.671256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1403/2776] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:06:56.675961 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.686106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e0c17-a97e-4d20-aa1b-d33cbb26f628 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.690885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.691534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.691822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.692099 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.692820 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.694469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.704989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1374/2777] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:06:56.709345 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.721054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9221ae-8e01-4cee-a0be-a480ca3d9733 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.725885 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.726862 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.727148 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.727427 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] PATH_INFO: `/v3/projects/25b3fac6c0af43ea99fd31e5a72935b6/users/319e91e5584541d0b9ada922ff610cdc/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.744622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b3fac6c0af43ea99fd31e5a72935b6, user_id=319e91e5584541d0b9ada922ff610cdc, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.747591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e697c9e-0fb7-498b-84a7-fe99c4647a1c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.778089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.833360 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1404/2778] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:06:56 2022] PUT /identity/v3/projects/25b3fac6c0af43ea99fd31e5a72935b6/users/319e91e5584541d0b9ada922ff610cdc/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:06:56.835607 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa7721-9747-438a-b463-131b1fec5a31 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.845573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.845921 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.846487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.848442 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:56.876972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] MFA Rules not processed for user `319e91e5584541d0b9ada922ff610cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:56.878190 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.892732 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.893507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.893804 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.894075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.895973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.948404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1405/2779] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:06:56.957748 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.959431 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d89f5-1306-423b-991e-c307418a6234 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:56.960107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:56.961480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:56.971373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1375/2780] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:56.974349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96575b0a-f2cf-451d-9590-95e1d0b73ea2 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:56.978339 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:56.979115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:56.979521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:56.979894 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:56.989788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:56.991325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:57.000625 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1406/2781] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:59.810550 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:59.811078 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:59.811177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:59.828868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] MFA Rules not processed for user `319e91e5584541d0b9ada922ff610cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:59.835457 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:59.837010 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1376/2782] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:00.420951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:00.426934 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:00.457362 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:00.501731 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddee6bd-f13d-4334-b484-62d40db30fb7 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:00.507024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:00.507704 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:00.507992 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:00.508276 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:00.510161 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:00.523323 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:00.524970 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992266e-964f-4b8b-9b0a-ff7f35ae5171 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:00.536995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1407/2783] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:09.291373 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:09.292951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:09.365402 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf12ac-19d7-48fc-a157-c23f03e251b9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:09.370102 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:09.371031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:09.371320 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:09.371617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] PATH_INFO: `/v3/users/7fa48e71823b4cdc91989e3c33db9d76` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:09.372819 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa48e71823b4cdc91989e3c33db9d76)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:09.374800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:09.517568 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa48e71823b4cdc91989e3c33db9d76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:09.525018 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa48e71823b4cdc91989e3c33db9d76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:09.525693 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa48e71823b4cdc91989e3c33db9d76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:09.532769 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:09.538620 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa48e71823b4cdc91989e3c33db9d76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:09.620421 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa48e71823b4cdc91989e3c33db9d76'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:09.626924 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1377/2784] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:07:09 2022] DELETE /identity/v3/users/7fa48e71823b4cdc91989e3c33db9d76 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:09.631455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:09.631951 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:09.632049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:09.650043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:09.651515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:09.718838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1408/2785] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:09.729196 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:09.730615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:09.789725 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823e2c5-6428-4b5d-8196-321e86a09cc9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:09.793344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:09.793994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:09.794281 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:09.794563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:09.804236 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:09.805684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:09.815843 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1378/2786] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:10.234767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:10.236214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:10.256712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06795a7c-5314-4801-9fd1-ce4560f4cf19 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:10.266080 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:10.271980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:10.271980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:10.271980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] PATH_INFO: `/v3/projects/8f84476142c9430487a52d95cf1d49ab` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:10.271980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f84476142c9430487a52d95cf1d49ab)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:10.271980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:10.441167 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f84476142c9430487a52d95cf1d49ab', 'request_id': 'req-4d24b496-2332-4315-b52f-aac6c26a4b1d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.531746 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f84476142c9430487a52d95cf1d49ab', 'request_id': 'req-4d24b496-2332-4315-b52f-aac6c26a4b1d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.538538 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f84476142c9430487a52d95cf1d49ab', 'request_id': 'req-4d24b496-2332-4315-b52f-aac6c26a4b1d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.549553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] The token cache is being invalidate because project 8f84476142c9430487a52d95cf1d49ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:10.549841 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.550468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.552033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1409/2787] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:07:10 2022] DELETE /identity/v3/projects/8f84476142c9430487a52d95cf1d49ab => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:10.556062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:10.566184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a988a-dfb2-4625-be59-4ede1f029827 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:10.624378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:10.625165 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:10.625450 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:10.625730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] PATH_INFO: `/v3/users/def65052cb774692af13018ce62be5fa` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:10.626950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def65052cb774692af13018ce62be5fa)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:10.629523 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:10.831541 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def65052cb774692af13018ce62be5fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.839562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def65052cb774692af13018ce62be5fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.840269 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def65052cb774692af13018ce62be5fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.852794 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def65052cb774692af13018ce62be5fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.955136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-29369008-b138-48fe-8f11-d08a78bd77e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def65052cb774692af13018ce62be5fa'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:10.962443 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1379/2788] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:07:10 2022] DELETE /identity/v3/users/def65052cb774692af13018ce62be5fa => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:11.402350 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:11.403922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:11.417479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b496-2332-4315-b52f-aac6c26a4b1d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:11.461886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:11.479502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:11.480301 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:11.480587 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:11.480868 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] PATH_INFO: `/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:11.482091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8e3df8c1034013a7a8ec7fcb98f832)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:11.484033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:11.667397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8e3df8c1034013a7a8ec7fcb98f832', 'request_id': 'req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:11.746953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8e3df8c1034013a7a8ec7fcb98f832', 'request_id': 'req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:11.752796 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8e3df8c1034013a7a8ec7fcb98f832', 'request_id': 'req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:11.764629 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] The token cache is being invalidate because project da8e3df8c1034013a7a8ec7fcb98f832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:11.764920 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:11.765641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-3f16a7dd-dcf3-4481-8d9b-1e02c4d89520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:11.767302 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1410/2789] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:07:11 2022] DELETE /identity/v3/projects/da8e3df8c1034013a7a8ec7fcb98f832 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:11.772182 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:11.772648 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:11.772742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:11.780096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:11.791005 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:11.792227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:11.853405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1380/2790] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:11.861244 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:11.861530 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:11.861808 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:11.880969 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:11.882160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:11.890007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1411/2791] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:11.894346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:11.904617 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c93b5-b571-4399-be80-c79e7356a12b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:11.909292 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:11.909944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:11.910233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:11.911093 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:11.911827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:07:11.912143 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:11.913735 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:11.925105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1381/2792] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:07:11.929248 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:11.939090 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccc63e-1945-4f37-82e5-e6cf040c7f55 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:11.943931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:11.944573 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:11.952397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:11.952397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] PATH_INFO: `/v3/projects` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:11.952397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:11.952397 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.062912 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1412/2793] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:07:11 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:07:12.064264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:12.074936 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42968f1-ee8e-4c37-827e-54004b9d7b0c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:12.132834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:12.133540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.133828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.134158 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] PATH_INFO: `/v3/users` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.134829 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:12.136863 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.275535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1382/2794] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:07:12 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:07:12.279652 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:12.289846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c38535-6a8b-4969-9c39-e52f9f30b28b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:12.294892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:12.295543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.295847 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.296172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.296892 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:12.298490 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.308860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1413/2795] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:07:12.313346 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:12.324266 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8fdc0-80a2-48d8-955d-97f681fcd21b admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:12.329046 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:12.329703 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.329991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.330272 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.331228 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:12.333615 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ec0d0-af6b-457d-b1b5-2b02bb9c96a3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.343902 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1383/2796] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:07:12.348042 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:12.357880 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8f00-ce72-4235-b3d4-1988391c9eb7 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:12.362386 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:12.363287 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.363598 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.363875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] PATH_INFO: `/v3/projects/a177f93d752a47d893da6989db08a99e/users/e700894113df4af99e9edb3071a5f38d/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.378187 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a177f93d752a47d893da6989db08a99e, user_id=e700894113df4af99e9edb3071a5f38d, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:12.381379 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.461367 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1414/2797] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:07:12 2022] PUT /identity/v3/projects/a177f93d752a47d893da6989db08a99e/users/e700894113df4af99e9edb3071a5f38d/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:12.466514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.466886 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.467215 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.485629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:12.486852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:12.546176 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1384/2798] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:12.555837 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:12.557545 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:12.616881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e048-bbea-4144-8f9f-b7de9154a3d4 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:12.620697 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:12.621369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:12.621653 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:12.621931 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:12.631935 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:12.633413 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fe07f-10d0-4b69-a055-3b5ef9c478cf service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:12.642735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1415/2799] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:13.224146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:13.225652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:13.240844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:13.308925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb32a4-fc95-4dd1-b0eb-05e38e039bbf None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:13.316865 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:13.317539 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:13.317830 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:13.318142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:13.321008 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:13.336254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:13.338202 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:13.353104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1385/2800] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:14.294107 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2048ad-dbe7-4be6-8f36-0e3843c5a613 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:14.294726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2048ad-dbe7-4be6-8f36-0e3843c5a613 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:14.294726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2048ad-dbe7-4be6-8f36-0e3843c5a613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:14.314214 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:14.323382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-7c2048ad-dbe7-4be6-8f36-0e3843c5a613 None None] MFA Rules not processed for user `e700894113df4af99e9edb3071a5f38d`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:14.325075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7c2048ad-dbe7-4be6-8f36-0e3843c5a613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:14.406470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1416/2801] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:14.417020 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:14.418555 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:14.514056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eaf084-f80b-4ad2-bea9-7b3f68b1f12e service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:14.517964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:14.518613 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:14.518927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:14.519250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:14.535175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:14.537307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:14.552736 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1386/2802] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:17.297169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d348b-ea51-4b99-859f-d3b3659ada88 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:17.297741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d348b-ea51-4b99-859f-d3b3659ada88 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:17.297741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d348b-ea51-4b99-859f-d3b3659ada88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:17.317596 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b85d348b-ea51-4b99-859f-d3b3659ada88 None None] MFA Rules not processed for user `e700894113df4af99e9edb3071a5f38d`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:17.318899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b85d348b-ea51-4b99-859f-d3b3659ada88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:17.326092 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1417/2803] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:17.565163 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:17.566659 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:17.578515 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac553-7de8-4c56-8681-52b8b7b7dfbf service cinder] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:17.582092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:17.582762 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:17.583050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:17.583330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:17.585021 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:17.594207 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:17.595726 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:17.604169 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1387/2804] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:22.145505 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f2884-e3b8-4a41-bc62-a5d7d91ebba0 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:22.145971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f2884-e3b8-4a41-bc62-a5d7d91ebba0 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:22.145971 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f2884-e3b8-4a41-bc62-a5d7d91ebba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:22.164207 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f82f2884-e3b8-4a41-bc62-a5d7d91ebba0 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:22.165411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f82f2884-e3b8-4a41-bc62-a5d7d91ebba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:22.227816 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1418/2805] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:22.354401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:22.356294 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:22.446337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a994e-d591-4a79-8fd7-fdda3cc58473 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:22.451592 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:22.452250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:22.452542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:22.452826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:22.460352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=100 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:22.471946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:22.471946 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:22.477029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1388/2806] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:07:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:07:23.414989 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a669c-0fe4-4247-b4c9-b97062f91b6f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:23.415443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a669c-0fe4-4247-b4c9-b97062f91b6f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:23.415553 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a669c-0fe4-4247-b4c9-b97062f91b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:23.432851 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-225a669c-0fe4-4247-b4c9-b97062f91b6f None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:23.434041 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-225a669c-0fe4-4247-b4c9-b97062f91b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:23.442033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1419/2807] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 15:07:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:23.453897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:23.464965 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e214fdf-daf4-4240-acaf-23cd02cb1716 service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:23.469945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:23.470581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:23.470897 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:23.471180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:23.479238 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:23.484401 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:23.488834 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1389/2808] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:07:24.110906 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f8f4-373f-4a79-9e2c-e5d54a355871 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:24.111196 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f8f4-373f-4a79-9e2c-e5d54a355871 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:24.111476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f8f4-373f-4a79-9e2c-e5d54a355871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:24.129728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3fa0f8f4-373f-4a79-9e2c-e5d54a355871 None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:24.131055 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3fa0f8f4-373f-4a79-9e2c-e5d54a355871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:24.138166 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1420/2809] 10.222.0.74 () {58 vars in 1009 bytes} [Wed May 25 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:24.150657 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:24.161600 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf5576-1377-4c9d-ae4a-ab98ae141fc3 service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:24.166590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:24.167248 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:24.167534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:24.167810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:24.176964 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:24.178623 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:24.187178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1390/2810] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:07:24.738147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:24.738666 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:24.738764 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:24.756611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] MFA Rules not processed for user `a0f1db595d804131bf47d7ae396d5889`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:24.757806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:24.765025 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1421/2811] 10.222.0.74 () {58 vars in 967 bytes} [Wed May 25 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:24.777530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:24.788828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898eec-aa4d-4183-af5d-71ebbaa131a4 service swift] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:24.794184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a7eff73b85406a8874547263816a68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:24.794862 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:24.795145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:24.795423 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:24.804108 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:24.807148 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea00f8a-bca2-49c6-8ce1-5c96af27771a service swift] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:24.814227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1391/2812] 10.222.0.74 () {60 vars in 1384 bytes} [Wed May 25 15:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:07:27.823916 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:27.825392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:27.842469 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901969-91d8-48e9-baa8-f3c1898e448d None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:27.844937 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:27.845806 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:27.846096 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:27.846375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] PATH_INFO: `/v3/users/319e91e5584541d0b9ada922ff610cdc` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:27.847641 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=319e91e5584541d0b9ada922ff610cdc)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:27.849634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:28.048823 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319e91e5584541d0b9ada922ff610cdc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.060310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319e91e5584541d0b9ada922ff610cdc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.060310 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319e91e5584541d0b9ada922ff610cdc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.070377 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319e91e5584541d0b9ada922ff610cdc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.072206 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:28.163247 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319e91e5584541d0b9ada922ff610cdc'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.169668 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1422/2813] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:07:27 2022] DELETE /identity/v3/users/319e91e5584541d0b9ada922ff610cdc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:28.174264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:28.174797 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:28.174921 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:28.193531 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:28.194772 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:28.252680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1392/2814] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:28.263066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:28.264478 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:28.326815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99902636-5381-40de-9d77-d06b7436d938 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:28.330788 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:28.331438 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:28.331722 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:28.332001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:28.341572 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:28.343034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:28.352772 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1423/2815] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:28.585681 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:28.587280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:28.599716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec88a1-a368-45a4-849e-e32e6edfdb42 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:28.604718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:28.605584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:28.610628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:28.610628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] PATH_INFO: `/v3/projects/25b3fac6c0af43ea99fd31e5a72935b6` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:28.610628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b3fac6c0af43ea99fd31e5a72935b6)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:28.610628 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:28.746267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b3fac6c0af43ea99fd31e5a72935b6', 'request_id': 'req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.840367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b3fac6c0af43ea99fd31e5a72935b6', 'request_id': 'req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.847354 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b3fac6c0af43ea99fd31e5a72935b6', 'request_id': 'req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.860562 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] The token cache is being invalidate because project 25b3fac6c0af43ea99fd31e5a72935b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:28.860832 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.861491 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6f027b5a-4acf-48a2-a82d-0f6a25ab1664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:28.863237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1393/2816] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:07:28 2022] DELETE /identity/v3/projects/25b3fac6c0af43ea99fd31e5a72935b6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:38.348359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=28 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:38.854842 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:29.559027 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:29.561468 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:29.652186 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c1225-2e23-445d-80a0-a921dfd9d0be service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:29.714560 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:29.715236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:29.715575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:29.715825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:29.717766 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:29.729460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:29.731126 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:29.745067 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1424/2817] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:08:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:39.737341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=67 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:42.396002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:42.396537 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:42.396651 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:42.414679 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] MFA Rules not processed for user `e700894113df4af99e9edb3071a5f38d`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:42.415926 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:42.471622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1394/2818] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:08:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:42.707068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:42.708543 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:42.762698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0123cd-7344-4f08-b143-135d81fe9742 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:42.766922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:42.767559 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:42.767839 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:42.768109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:42.769860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:42.779522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:42.781005 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:42.789122 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1425/2819] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:08:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:52.463747 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=25 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:52.785466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=26 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:57.001635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:57.003199 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:57.018828 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3eea9-06b4-4072-82fa-71d4a4dfb49e None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:57.024551 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:57.025276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:57.025767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:57.026120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:57.028388 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:57.040900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:57.043692 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:57.054057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1395/2820] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:09:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:57.137798 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:57.139312 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:57.209898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592400-9e70-40e1-8262-66f1ce0aa756 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:57.214523 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:57.215455 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:57.215783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:57.216089 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] PATH_INFO: `/v3/users/e700894113df4af99e9edb3071a5f38d` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:57.217294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e700894113df4af99e9edb3071a5f38d)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:57.219420 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:57.384570 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e700894113df4af99e9edb3071a5f38d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:57.391373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e700894113df4af99e9edb3071a5f38d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:57.392048 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e700894113df4af99e9edb3071a5f38d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:57.404393 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e700894113df4af99e9edb3071a5f38d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:57.459387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-d64d7b54-278d-4b64-8e29-0433b48afb73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e700894113df4af99e9edb3071a5f38d'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:57.473710 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1426/2821] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:09:57 2022] DELETE /identity/v3/users/e700894113df4af99e9edb3071a5f38d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:57.723530 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:57.736655 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920248-1e2e-4a55-ba26-edf6260b30a2 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:57.798375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:57.799287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:57.800124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:57.800405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] PATH_INFO: `/v3/projects/a177f93d752a47d893da6989db08a99e` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:57.801635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a177f93d752a47d893da6989db08a99e)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:57.803595 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:57.943210 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a177f93d752a47d893da6989db08a99e', 'request_id': 'req-fb863160-b0cd-4047-9a2b-01984ade82a6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:58.021887 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a177f93d752a47d893da6989db08a99e', 'request_id': 'req-fb863160-b0cd-4047-9a2b-01984ade82a6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:58.028712 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a177f93d752a47d893da6989db08a99e', 'request_id': 'req-fb863160-b0cd-4047-9a2b-01984ade82a6'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:58.040055 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] The token cache is being invalidate because project a177f93d752a47d893da6989db08a99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:09:58.040344 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:58.041009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-fb863160-b0cd-4047-9a2b-01984ade82a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:58.042590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1396/2822] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:09:57 2022] DELETE /identity/v3/projects/a177f93d752a47d893da6989db08a99e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:58.047771 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.048066 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.048347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.067780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:58.069191 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:58.129803 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1427/2823] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:58.137920 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.138216 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.138505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.156933 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:58.158263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:58.166811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1397/2824] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:58.171141 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.181575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8097d3-554c-4865-94d3-794607c233b3 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.186476 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.187162 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.187449 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.187730 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.188488 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:09:58.188799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.190399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.202147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1428/2825] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:09:58.206503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.215787 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26fd7b-5e3f-4cb8-b81b-1cdf836733cb None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.220499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.221699 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.222275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.222553 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.223280 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.225184 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.294912 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1398/2826] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:09:58 2022] POST /identity/v3/projects => generated 389 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:58.299104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.309131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e6625-880d-440d-8169-267b4b2e5198 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.313082 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:58.365829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.366503 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.366817 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.367109 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.367786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.369739 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.439609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1429/2827] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:09:58 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:58.443904 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.454181 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9dafa-c0c9-4e80-bc5d-92e7f269d691 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.459598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.460372 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.460662 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.461060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.461792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.463779 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.474991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1399/2828] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:58.479172 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.489139 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb7c89-e507-41f0-afde-acffb6e9acba admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.494095 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.494802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.495093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.495373 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.496104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.497712 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346628-cf89-47f4-acdd-055ab646bd10 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.507804 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1430/2829] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:58.512069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.522711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c44e7b-c45a-4605-ba7e-a093b59c4a13 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.527326 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.528190 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.528472 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.528746 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] PATH_INFO: `/v3/projects/89a169976a7c453faa544cb6f78a5802/users/4c0dd594e7b949599a5600bdd51f5412/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.543653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a169976a7c453faa544cb6f78a5802, user_id=4c0dd594e7b949599a5600bdd51f5412, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.546352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.634581 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1400/2830] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:09:58 2022] PUT /identity/v3/projects/89a169976a7c453faa544cb6f78a5802/users/4c0dd594e7b949599a5600bdd51f5412/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:09:58.644719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.645013 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.645299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.673316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] MFA Rules not processed for user `4c0dd594e7b949599a5600bdd51f5412`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:58.675227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:58.743491 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1431/2831] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:09:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:58.754367 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:58.756140 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:58.770786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:58.826037 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76cbe4-b64c-4efe-b340-d77a1b670e3c admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:58.829695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:58.830360 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:58.830683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:58.830973 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:58.839436 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:58.840885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115817e-abdd-40fe-918c-d9271e2abfef service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:58.850868 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1401/2832] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:09:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:08.740060 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:08.876362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=20 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:22.791717 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:22.856435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa5b1-9b24-4e58-b480-68479bd4668f None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:22.861343 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:22.862209 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:22.862525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:22.862863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] PATH_INFO: `/v3/users/4c0dd594e7b949599a5600bdd51f5412` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:22.864123 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0dd594e7b949599a5600bdd51f5412)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:22.866056 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:23.115882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0dd594e7b949599a5600bdd51f5412'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:23.122626 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0dd594e7b949599a5600bdd51f5412'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:23.123338 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0dd594e7b949599a5600bdd51f5412'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:23.136104 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0dd594e7b949599a5600bdd51f5412'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:23.287531 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0dd594e7b949599a5600bdd51f5412'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:23.294125 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1432/2833] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:10:22 2022] DELETE /identity/v3/users/4c0dd594e7b949599a5600bdd51f5412 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:10:23.298742 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:23.299722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:23.299722 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:23.317893 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:23.319147 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:23.380626 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1402/2834] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:23.390383 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:23.391948 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:23.450160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d2a2-aaf4-45bd-a05c-b1c5cb11603d admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:23.454609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:23.455879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:23.455879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:23.455879 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:23.465838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:23.467337 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8385951-413d-4b83-a9db-6b26188ba8a3 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:23.477418 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1433/2835] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:23.884164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:23.885590 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:23.898377 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec407a-5fd8-4321-92e8-43da247d1bb7 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:23.903092 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:23.906532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:23.906532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:23.906532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] PATH_INFO: `/v3/projects/89a169976a7c453faa544cb6f78a5802` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:23.906532 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a169976a7c453faa544cb6f78a5802)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:23.907656 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.018288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a169976a7c453faa544cb6f78a5802', 'request_id': 'req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:24.084031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a169976a7c453faa544cb6f78a5802', 'request_id': 'req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:24.091050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a169976a7c453faa544cb6f78a5802', 'request_id': 'req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:24.102405 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] The token cache is being invalidate because project 89a169976a7c453faa544cb6f78a5802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:24.102728 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:24.103413 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-5a9f4ae3-dedd-4c98-aa8f-909c3c2141bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:24.104994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1403/2836] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:10:23 2022] DELETE /identity/v3/projects/89a169976a7c453faa544cb6f78a5802 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:24.110033 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.110513 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.110621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.131288 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:24.132497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:24.150507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:24.195467 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1434/2837] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:24.203485 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.203781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.204061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.223101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:24.224328 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:24.232337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1404/2838] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:24.236642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.247253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3f1c-5122-4668-8001-54617be42c78 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.251932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.258726 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.268863 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1435/2839] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:10:24.273711 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.282867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce14bb-23a8-4875-8c62-08e01f3c7c1b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.287488 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.288127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.288410 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.288688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.289378 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.291295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.418900 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1405/2840] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:10:24 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:24.423320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.434135 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c6ea-8d04-431f-8a02-7715dc81959c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.497194 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.497871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.498160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.498470 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.499203 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.501202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.605991 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1436/2841] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:10:24 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:10:24.609950 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.621271 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55a299-7b5a-4533-b9a6-6899ac363f90 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.626573 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.629384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.629384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.629384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.629384 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.631048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.642619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1406/2842] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:24.646576 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.657047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d695060-7357-4ed4-9e8c-bd4dbae99b11 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.661741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.662394 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.662718 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.663004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.663732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.665322 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c3144-d2e1-4e63-83c7-a38cbc93c0f0 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.675871 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1437/2843] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:10:24.680096 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.692204 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b12f2c-5f5c-40eb-abf6-03fa8d69ec49 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.697880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:24.699033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.699391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.699766 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] PATH_INFO: `/v3/projects/8946e4ac08474e979c6f09a449fd4e19/users/4e4e2ece8d2f40b38341da610f7ca82c/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.718104 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8946e4ac08474e979c6f09a449fd4e19, user_id=4e4e2ece8d2f40b38341da610f7ca82c, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:24.721011 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:24.820680 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1407/2844] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:10:24 2022] PUT /identity/v3/projects/8946e4ac08474e979c6f09a449fd4e19/users/4e4e2ece8d2f40b38341da610f7ca82c/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:24.831967 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:24.832300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:24.832591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:24.859938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] MFA Rules not processed for user `4e4e2ece8d2f40b38341da610f7ca82c`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:24.861174 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:24.924226 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1438/2845] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:24.936554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:24.938250 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:24.959786 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:24.995501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a289d-cb51-4b3b-b1b7-5a9948dadbbe admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:24.999554 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:25.000323 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:25.000499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:25.000781 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:25.009041 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:25.010951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990cfd1-89f8-49d4-9707-454efbfb7e87 service cinder] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:25.019291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1408/2846] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:10:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:34.921331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:35.070476 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=17 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:35.152522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.213293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fec8-a7ab-42a8-be2e-ff11dc94fa12 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.217811 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.218778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.219069 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.219347 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] PATH_INFO: `/v3/users/4e4e2ece8d2f40b38341da610f7ca82c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.220525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4e2ece8d2f40b38341da610f7ca82c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.222443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.417752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4e2ece8d2f40b38341da610f7ca82c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.424563 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4e2ece8d2f40b38341da610f7ca82c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.425236 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4e2ece8d2f40b38341da610f7ca82c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.437306 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4e2ece8d2f40b38341da610f7ca82c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.532677 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4e2ece8d2f40b38341da610f7ca82c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.538961 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1439/2847] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:10:35 2022] DELETE /identity/v3/users/4e4e2ece8d2f40b38341da610f7ca82c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:35.543645 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.546052 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.562031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:35.563276 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:35.622810 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1409/2848] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:35.632860 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.634279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:35.692700 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0b51-92ea-4b2b-b55b-dc5dcb512c95 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.696352 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.697012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.697281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.697562 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.706537 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.708004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95a56a-d039-4228-ac5c-947237edffc2 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.717567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1440/2849] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:36.161060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.161060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:36.169466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e05913-1c02-40bb-99a3-1adad70d426b None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.174237 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.175133 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.175415 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.175691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] PATH_INFO: `/v3/projects/8946e4ac08474e979c6f09a449fd4e19` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.177570 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8946e4ac08474e979c6f09a449fd4e19)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.179538 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.283168 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8946e4ac08474e979c6f09a449fd4e19', 'request_id': 'req-1d656414-9584-46c3-b188-186bf97984a3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:36.367391 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8946e4ac08474e979c6f09a449fd4e19', 'request_id': 'req-1d656414-9584-46c3-b188-186bf97984a3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:36.375381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8946e4ac08474e979c6f09a449fd4e19', 'request_id': 'req-1d656414-9584-46c3-b188-186bf97984a3'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:36.387330 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] The token cache is being invalidate because project 8946e4ac08474e979c6f09a449fd4e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:36.387669 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:36.389303 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-1d656414-9584-46c3-b188-186bf97984a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:36.389992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1410/2850] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:10:36 2022] DELETE /identity/v3/projects/8946e4ac08474e979c6f09a449fd4e19 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:36.395589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.396099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.396199 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.413674 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:36.414833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:36.432044 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:36.474765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1441/2851] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:36.483066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.483356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.483682 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.502652 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:36.503994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:36.512816 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1411/2852] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:36.516698 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.526801 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c830248-6c57-4b2c-948e-a3cae3bdd1e6 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.531480 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.532183 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.532487 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.532882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.533622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:10:36.533941 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.535592 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.547938 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1442/2853] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:10:36.552939 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.562608 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9005c4-7bfa-4f93-a68c-319bd90e317d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.567898 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.568546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.568827 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.569101 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.569785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.571716 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.698890 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1412/2854] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:10:36 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:36.703156 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.713599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f298db-65ce-4c50-9921-12ec72f22abf admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.770582 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.771525 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.771649 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.771846 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.772554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.774532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.878973 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1443/2855] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:10:36 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:36.883563 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.894068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b081-353e-4e37-8c2a-cdd1a419043e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.898969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.899633 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.900057 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.900345 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.901076 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.903397 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.916424 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1413/2856] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:36.920474 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.931813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452838a-ee4c-43bc-a629-82aed5816ad2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.936735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.937399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.937682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.937972 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.938713 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.940303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6c878-bbbb-4310-8868-66f4ca3ceb97 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:36.950929 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1444/2857] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:36.955167 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:36.965406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226720-4288-4be6-bd33-3d411c6813d6 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:36.970050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:36.970989 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:36.971255 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:36.971534 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] PATH_INFO: `/v3/projects/6405d7039f25476f81b98903d2af54f2/users/cc778dde80744f318367d27fde29d47c/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:36.986918 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6405d7039f25476f81b98903d2af54f2, user_id=cc778dde80744f318367d27fde29d47c, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:36.990116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:37.062406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1414/2858] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:10:36 2022] PUT /identity/v3/projects/6405d7039f25476f81b98903d2af54f2/users/cc778dde80744f318367d27fde29d47c/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:37.067875 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73afe1-d02a-4106-8d30-a12f0828fb7e None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:37.068169 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73afe1-d02a-4106-8d30-a12f0828fb7e None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:37.068452 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73afe1-d02a-4106-8d30-a12f0828fb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:37.090522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8d73afe1-d02a-4106-8d30-a12f0828fb7e None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:37.092532 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8d73afe1-d02a-4106-8d30-a12f0828fb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:37.159001 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1445/2859] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:37.169775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:37.171598 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:37.194286 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:37.231365 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f81f3-8f66-43dd-9e47-6bb8b06141fd admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:37.235045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:37.235683 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:37.235967 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:37.236245 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:37.246284 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:37.250082 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:37.257505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1415/2860] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:42.627813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:42.628357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:42.628357 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:42.657962 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] MFA Rules not processed for user `cc778dde80744f318367d27fde29d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:42.659227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:42.677387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:42.730119 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1446/2861] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:42.735030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:42.745991 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17cb8f-85e9-4090-a473-675ca9a09675 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:42.750984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:42.751870 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:42.752164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:42.752444 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:42.753152 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:42.755291 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:42.852624 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1416/2862] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:10:42 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:42.856738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:42.867882 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54c3b-f01e-41fa-83e2-99b0528a35f4 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:42.940376 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:42.941609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:42.942472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:42.942799 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:42.943524 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:42.945568 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:43.071518 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1447/2863] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:10:42 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:10:43.075773 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:43.088025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258198b2-e12e-4f7e-b50d-89f36c0e1736 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:43.093256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:43.095925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:43.095925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:43.095925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:43.095925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:43.097120 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:43.109100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1417/2864] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:43.114462 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:43.125296 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2fc4c-b2dc-40b2-bc97-59879563feaf admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:43.130612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:43.131540 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:43.131822 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:43.132099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] PATH_INFO: `/v3/projects/5569e2e8eba84c4ea505cf9480e70b12/users/1e9b17d4bb99420fb862e44672b1b0d8/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:43.149968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5569e2e8eba84c4ea505cf9480e70b12, user_id=1e9b17d4bb99420fb862e44672b1b0d8, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:43.152924 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:43.285267 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1448/2865] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:10:43 2022] PUT /identity/v3/projects/5569e2e8eba84c4ea505cf9480e70b12/users/1e9b17d4bb99420fb862e44672b1b0d8/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:43.289256 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:43.300222 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2ef4-61b9-4f9d-b1b2-47b3eb87403e admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:43.364889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:43.365665 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:43.366022 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:43.366369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:43.367208 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:43.368888 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fabc13-5539-4e95-9d0b-0a1f28322870 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:43.380412 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1418/2866] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:43.384634 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:43.395499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749da54-04d7-45e1-a490-2544ed1ab7e7 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:43.400381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:43.401294 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:43.401580 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:43.401858 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] PATH_INFO: `/v3/projects/5569e2e8eba84c4ea505cf9480e70b12/users/1e9b17d4bb99420fb862e44672b1b0d8/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:43.404047 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5569e2e8eba84c4ea505cf9480e70b12, user_id=1e9b17d4bb99420fb862e44672b1b0d8, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:43.406738 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:43.418211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1449/2867] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:10:43 2022] PUT /identity/v3/projects/5569e2e8eba84c4ea505cf9480e70b12/users/1e9b17d4bb99420fb862e44672b1b0d8/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:49.606072 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf6ef-0e70-4c26-9357-2833da0f0fdd None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:49.606676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf6ef-0e70-4c26-9357-2833da0f0fdd None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:49.606676 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbf6ef-0e70-4c26-9357-2833da0f0fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:49.637653 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-f6bbf6ef-0e70-4c26-9357-2833da0f0fdd None None] MFA Rules not processed for user `1e9b17d4bb99420fb862e44672b1b0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:49.638775 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f6bbf6ef-0e70-4c26-9357-2833da0f0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:49.752811 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1419/2868] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:49.773204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:49.779173 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:49.859852 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:49.895824 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536bde3-67bc-43cc-b19d-17ff4946e6b8 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:49.901705 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:49.906883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:49.906883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:49.906883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:49.964923 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:49.966558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:49.976907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1450/2869] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:10:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:52.637289 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:52.637845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:52.637845 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:52.656016 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] MFA Rules not processed for user `cc778dde80744f318367d27fde29d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:52.657283 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:52.664178 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1420/2870] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:59.972046 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=36 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:02.650741 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:13.592757 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:13.594112 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:13.658554 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3363370-ff85-4660-8f3d-4e5a8f9431e1 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:13.663152 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:13.663792 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:13.664078 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:13.664359 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:13.666264 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:13.680642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:13.682093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:13.694024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1451/2871] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:15.757782 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:15.759296 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:15.823940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c7704-2ee3-45a5-bba9-873c878d84a5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:15.827611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:15.828382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:15.828670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:15.828956 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:15.830880 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:15.845221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:15.846694 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:15.858999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1421/2872] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:19.279763 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:19.290031 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dff555-a2e1-40b0-b7a6-80dbbbd06116 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:19.299106 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:19.299767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:19.300404 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:19.301235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:19.303121 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:19.314888 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:19.317192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:19.326756 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1452/2873] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:20.313962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:20.315752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:20.341969 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef82e3-0176-4a50-85c2-0234d022e512 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:20.351155 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:20.353684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:20.353684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:20.353684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:20.355337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:20.371727 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:20.373353 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:20.388000 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1422/2874] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:29.323227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=32 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:30.382793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=32 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:33.539988 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:33.548744 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4ac5b-b6b7-446b-8119-95505297e8d0 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:33.552457 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:33.553115 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:33.553492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:33.553774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:33.562966 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:33.564392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:33.573770 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1453/2875] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:35.826627 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:35.838301 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c14cd-11f5-46cb-ab4a-88029ca29528 service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:35.843799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:35.844487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:35.844718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:35.844994 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:35.846809 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:35.859844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:35.861509 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:35.883034 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1423/2876] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:11:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:43.569616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:45.867885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:13:45.897791 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:13:45.899239 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:13:45.968200 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978df0ae-45de-4ccb-8d16-73859ab33645 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:13:45.973369 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:13:45.974021 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:13:45.974308 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:13:45.974589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:13:45.976544 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:13:45.988464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:13:45.990075 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:13:46.000774 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1454/2877] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:13:55.996441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=32 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:14:05.565474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:05.576356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf1ed3-e74c-4aba-8c8b-ad420916047b service nova] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:05.580466 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:05.580605 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:05.580826 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:05.581110 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:05.583146 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:05.594017 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:05.595492 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:05.603992 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1424/2878] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:14:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:05.687459 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:05.698068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec60d4-5d55-40cb-850c-b594409fda3c service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:05.701691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:05.702330 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:05.702619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:05.702927 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:05.704709 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:05.718114 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:05.719591 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:05.731007 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1455/2879] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:14:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:05.793962 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:05.805999 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bcad0-df0d-4d5a-ae32-9c0a999afa12 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:05.811839 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:05.812467 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:05.812750 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:05.813025 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:05.814793 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:05.826756 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:05.828366 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:05.838166 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1425/2880] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:14:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:07.419670 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:07.435997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2b7b-6785-4121-8670-9ef301efb517 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:07.442349 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:07.443034 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:07.443316 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:07.443678 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:07.445898 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:07.459711 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:07.461363 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:07.472886 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1456/2881] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:15.833958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=12 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:14:17.468285 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=12 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:15:58.106941 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:15:58.116875 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906580b-c7a7-4a95-b12b-d18a00aa70b4 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:15:58.120481 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:15:58.121116 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:15:58.121406 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:15:58.121684 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:15:58.130535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:15:58.131986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:15:58.140760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1426/2882] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:04.486099 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:04.495752 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a228af-3cd7-4435-9e82-db456e9f627b service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:04.499384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:04.500018 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:04.500307 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:04.500588 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:04.561341 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:04.562869 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:04.572432 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1457/2883] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:08.060390 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:08.061861 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:08.077075 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a05b6f-8169-432a-bc9e-967efdea1974 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:08.082906 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:08.083557 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:08.084073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:08.084584 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:08.086400 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:08.099227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:08.100825 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:08.111381 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1427/2884] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:12.019780 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:12.059299 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59881199-5172-430a-883b-5fa0c4e67b81 service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:12.064497 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:12.065375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:12.065661 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:12.065944 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] PATH_INFO: `/v3/users/cc778dde80744f318367d27fde29d47c` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:12.067204 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc778dde80744f318367d27fde29d47c)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:12.069147 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:12.344293 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc778dde80744f318367d27fde29d47c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:12.351767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc778dde80744f318367d27fde29d47c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:12.352443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc778dde80744f318367d27fde29d47c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:12.364381 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc778dde80744f318367d27fde29d47c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:12.495443 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc778dde80744f318367d27fde29d47c'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:12.501609 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1458/2885] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:16:12 2022] DELETE /identity/v3/users/cc778dde80744f318367d27fde29d47c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:12.812246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:12.852295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8694f-f399-4746-ae1f-77f51fc16c7c service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:12.910362 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:12.911172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:12.911456 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:12.911731 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] PATH_INFO: `/v3/projects/6405d7039f25476f81b98903d2af54f2` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:12.912907 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6405d7039f25476f81b98903d2af54f2)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:12.914835 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:13.059369 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6405d7039f25476f81b98903d2af54f2', 'request_id': 'req-c221f9fd-b5dc-4588-9abe-d59acad09cb1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.163930 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6405d7039f25476f81b98903d2af54f2', 'request_id': 'req-c221f9fd-b5dc-4588-9abe-d59acad09cb1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.170577 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6405d7039f25476f81b98903d2af54f2', 'request_id': 'req-c221f9fd-b5dc-4588-9abe-d59acad09cb1'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.194753 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] The token cache is being invalidate because project 6405d7039f25476f81b98903d2af54f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:16:13.195045 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.195670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.197268 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1428/2886] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:16:12 2022] DELETE /identity/v3/projects/6405d7039f25476f81b98903d2af54f2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:13.201271 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:13.202743 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:13.225151 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13942dcc-fa60-437e-9b9d-e1ac5085c73b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:13.255767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:13.281619 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:13.283129 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:13.283412 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:13.283686 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] PATH_INFO: `/v3/users/1e9b17d4bb99420fb862e44672b1b0d8` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:13.284890 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9b17d4bb99420fb862e44672b1b0d8)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:13.286838 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:13.489802 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9b17d4bb99420fb862e44672b1b0d8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.496564 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9b17d4bb99420fb862e44672b1b0d8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.497227 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9b17d4bb99420fb862e44672b1b0d8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.509372 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9b17d4bb99420fb862e44672b1b0d8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.579088 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-9ca53ff4-c16c-492d-80ee-17f3a5893f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9b17d4bb99420fb862e44672b1b0d8'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:13.585389 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1459/2887] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:16:13 2022] DELETE /identity/v3/users/1e9b17d4bb99420fb862e44672b1b0d8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:13.828574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:13.830035 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:13.842979 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221f9fd-b5dc-4588-9abe-d59acad09cb1 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:13.870180 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:13.900803 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:13.901667 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:13.901958 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:13.902239 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] PATH_INFO: `/v3/projects/5569e2e8eba84c4ea505cf9480e70b12` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:13.903486 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5569e2e8eba84c4ea505cf9480e70b12)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:13.905416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.060634 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5569e2e8eba84c4ea505cf9480e70b12', 'request_id': 'req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:14.164503 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5569e2e8eba84c4ea505cf9480e70b12', 'request_id': 'req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:14.171350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5569e2e8eba84c4ea505cf9480e70b12', 'request_id': 'req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:14.181817 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] The token cache is being invalidate because project 5569e2e8eba84c4ea505cf9480e70b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:16:14.182106 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:14.182764 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-6be136f0-a0d9-4bf7-9aa4-ea4c389690e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:16:14.184319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1429/2888] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:16:13 2022] DELETE /identity/v3/projects/5569e2e8eba84c4ea505cf9480e70b12 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:14.189945 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.190241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.190521 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.210472 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:14.213977 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:14.270440 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1460/2889] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:14.278253 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.278546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.278885 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.296996 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:14.298221 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:14.306293 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1430/2890] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:14.310542 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.320825 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2734fe-b10a-4c97-af77-1e08ec2f3a53 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.325364 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.326017 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.326300 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.326578 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] PATH_INFO: `/v3/domains` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.327331 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:16:14.327750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.330012 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.341719 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1461/2891] 10.222.0.74 () {58 vars in 1152 bytes} [Wed May 25 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:16:14.346496 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.355670 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba225b2-0175-4ba9-bf0e-ad12affdeab9 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.361048 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.361701 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.361986 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.362264 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.363012 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.364944 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.494987 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1431/2892] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:16:14 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:16:14.499006 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.508995 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47666085-28af-4ff0-a837-de3aa05eee3b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.547623 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:14.566008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.566721 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.567008 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.567286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.567980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.569896 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.648256 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1462/2893] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:16:14 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:16:14.652287 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.663649 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f5e2-a720-4ecc-b97a-746678f7c1c9 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.668279 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.669066 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.669352 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.669629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.670356 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.671970 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.683319 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1432/2894] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:16:14.687425 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.698093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feeedb-a773-49e7-82f6-82bd168be06c admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.702663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.703325 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.703611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.703891 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] PATH_INFO: `/v3/roles` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.704611 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.706211 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a6332-7ea6-4c76-a951-b2d5572d64af admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.716622 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1463/2895] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:16:14.720789 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.730866 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce381d03-7d86-4963-9299-bc6a42186bbb admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:14.736194 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:14.737069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] REQUEST_METHOD: `PUT` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.737392 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.737640 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] PATH_INFO: `/v3/projects/9e5a54632992428ca774d47f3f20d6b9/users/d13d9b9d3e48455082a74eaa3f1b4f18/roles/3f2e67fc62664875876c5e7f51498b6d` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.752822 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5a54632992428ca774d47f3f20d6b9, user_id=d13d9b9d3e48455082a74eaa3f1b4f18, role_id=3f2e67fc62664875876c5e7f51498b6d)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:14.755561 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:14.819396 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1433/2896] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:16:14 2022] PUT /identity/v3/projects/9e5a54632992428ca774d47f3f20d6b9/users/d13d9b9d3e48455082a74eaa3f1b4f18/roles/3f2e67fc62664875876c5e7f51498b6d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:14.824533 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809d6b-4689-4fcc-ae88-5de9c943d195 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:14.824856 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809d6b-4689-4fcc-ae88-5de9c943d195 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:14.825143 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809d6b-4689-4fcc-ae88-5de9c943d195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:14.843631 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-8c809d6b-4689-4fcc-ae88-5de9c943d195 None None] MFA Rules not processed for user `e695952cd3d8473291efec1acc9a5e16`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:14.845507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8c809d6b-4689-4fcc-ae88-5de9c943d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:14.905642 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1464/2897] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:14.916001 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:14.917867 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:14.971156 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:15.005160 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3235c8-3627-4566-879b-b83c20726032 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:15.010105 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:15.011175 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:15.011638 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:15.012121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:15.024053 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:15.025487 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:15.035004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1434/2898] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:20.389624 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.390202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.390202 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:20.424569 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.auth.core [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] MFA Rules not processed for user `d13d9b9d3e48455082a74eaa3f1b4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=92306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:20.426014 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:20.506558 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1465/2899] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:20.511399 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:20.526267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8102f61-a4d0-4dc6-821a-dec7299d6e1c service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:20.531069 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:20.531730 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.532068 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.532348 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] PATH_INFO: `/v3/projects` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:20.533205 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:20.535164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:20.621094 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1435/2900] 10.222.0.74 () {60 vars in 1152 bytes} [Wed May 25 15:16:20 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:16:20.625213 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:20.637694 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8616-8339-4ab5-a3c6-6a68446d0e48 None None] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:20.684766 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:20.697249 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:20.697926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] REQUEST_METHOD: `POST` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.698210 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.698492 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] PATH_INFO: `/v3/users` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:20.699235 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:20.701281 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:20.773429 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1466/2901] 10.222.0.74 () {60 vars in 1143 bytes} [Wed May 25 15:16:20 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:16:20.777629 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:20.788009 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01badef-8d80-43e9-ba25-4297833419ee admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:20.793389 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:20.793535 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.793751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.794029 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:20.794802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:20.796411 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:20.808637 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1436/2902] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:16:20.813024 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:20.824098 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8f6-da54-42df-b90e-f9013cf212fd admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:20.829708 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:20.830593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.830960 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.831241 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] PATH_INFO: `/v3/projects/3b45574471b34bd9be7ec77f34c48f75/users/f834b244b18840989978b168d6dd28f2/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:20.848399 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b45574471b34bd9be7ec77f34c48f75, user_id=f834b244b18840989978b168d6dd28f2, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:20.851084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:20.924728 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1467/2903] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:16:20 2022] PUT /identity/v3/projects/3b45574471b34bd9be7ec77f34c48f75/users/f834b244b18840989978b168d6dd28f2/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:20.928767 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:20.939871 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5a3ef-6c8a-4d17-b941-532263ec513d admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:20.998337 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:20.999049 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:20.999338 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:20.999616 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] PATH_INFO: `/v3/roles` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:21.000350 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:21.002556 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d67f8-8327-41f0-918e-4a747dc58ced admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:21.015162 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1437/2904] 10.222.0.74 () {58 vars in 1121 bytes} [Wed May 25 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:16:21.018279 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:21.029036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20128-8dce-41ee-9412-688771d57de2 admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:21.034599 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:21.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:21.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:21.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] PATH_INFO: `/v3/projects/3b45574471b34bd9be7ec77f34c48f75/users/f834b244b18840989978b168d6dd28f2/roles/d38d6cda16454360a4f48025ab4df728` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:21.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b45574471b34bd9be7ec77f34c48f75, user_id=f834b244b18840989978b168d6dd28f2, role_id=d38d6cda16454360a4f48025ab4df728)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:21.041245 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:21.052758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1468/2905] 10.222.0.74 () {60 vars in 1482 bytes} [Wed May 25 15:16:21 2022] PUT /identity/v3/projects/3b45574471b34bd9be7ec77f34c48f75/users/f834b244b18840989978b168d6dd28f2/roles/d38d6cda16454360a4f48025ab4df728 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:16:26.677982 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba202-505e-45af-9551-dae9c5314ce7 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:26.678507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba202-505e-45af-9551-dae9c5314ce7 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:26.678507 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ba202-505e-45af-9551-dae9c5314ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:26.710127 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-3f5ba202-505e-45af-9551-dae9c5314ce7 None None] MFA Rules not processed for user `f834b244b18840989978b168d6dd28f2`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:26.711342 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-3f5ba202-505e-45af-9551-dae9c5314ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:26.807164 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1438/2906] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:26.825932 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:26.827775 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:26.910522 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eac15-578e-4232-89a4-a55d14a8b5bf admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:26.916411 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:26.917068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:26.917353 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:26.917627 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:26.977786 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:26.979445 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:26.990682 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1469/2907] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:36.803790 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:36.984464 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:51.783529 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:51.784033 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:51.784151 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:51.801619 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] MFA Rules not processed for user `d13d9b9d3e48455082a74eaa3f1b4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:16:51.802923 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:51.810227 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1439/2908] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:16:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:16:51.848320 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:51.849732 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:51.908919 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc20df-a978-462e-9a61-2e116200aec1 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:51.912575 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:51.913221 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:51.913504 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:51.913783 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:51.915612 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:51.935054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:51.935054 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:51.942959 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1470/2909] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:52.221540 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:52.223510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:52.288469 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28b7e-5888-46f4-b9b8-7ee25359f311 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:52.292056 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:52.292697 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:52.292985 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:52.293267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:52.295061 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:52.304814 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:52.306505 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:52.314738 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1440/2910] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:53.073720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:53.084638 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664d53-b414-4bc4-be9a-e9319b08466e service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:53.088446 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:53.089094 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:53.089378 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:53.089657 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:53.091579 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:53.107911 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:53.109767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:53.122813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1471/2911] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:56.360023 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:56.369267 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384b3c9-3e56-4c24-a084-4b4f2f7662ab service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:56.373307 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:56.373957 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:56.374242 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:56.374518 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:56.378536 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:56.387416 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:56.389031 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:56.398145 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1441/2912] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:57.237735 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:57.250845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c796c-a9eb-40ac-956b-dabfa1de5e9a service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:57.257767 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:57.258431 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:57.258812 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:57.259093 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:57.260909 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:16:57.275068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:57.277009 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:57.291765 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1472/2913] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:16:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:17:06.393569 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=39 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:17:07.283602 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=38 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:17:13.042760 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:17:13.052206 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8aa2e-c776-4934-a652-6679e87126c1 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:17:13.056246 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:17:13.056889 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:17:13.057177 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:17:13.057447 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:17:13.067375 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:17:13.068800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5557c00-3af5-4496-881b-d11798f86e73 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:17:13.078153 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1442/2914] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:17:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:17:23.074060 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:51.855922 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:51.857375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:51.944810 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510df44-2734-4191-ba68-3bb673702b14 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:51.949845 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:51.950489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:51.950813 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:51.951097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:51.952881 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:51.964703 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:51.966283 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:51.976068 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1473/2915] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:18:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:01.971836 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=41 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:35.650136 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:35.650951 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:35.651142 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:35.670695 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] MFA Rules not processed for user `d13d9b9d3e48455082a74eaa3f1b4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:35.671927 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:35.679288 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1443/2916] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:37.097753 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:37.118253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc8d5-1868-42d8-955c-6c2815db420d service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:37.126593 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:37.127900 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:37.128466 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:37.129028 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:37.132036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:37.148760 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:37.151663 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:37.164192 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1474/2917] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:45.664050 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=3 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:47.158392 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:53.287948 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:53.289522 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:53.308752 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b458a6-abec-4acf-86c3-02740dafd1f5 None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:53.314382 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:53.315089 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:53.315434 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:53.315763 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:53.317558 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:53.330611 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:53.334499 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd63b57-9905-4b79-a3f1-dda1f0830ef5 service nova] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:53.344332 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1444/2918] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:03.339032 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:21:48.548170 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:21:48.549691 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:21:48.585084 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1b1d-23fc-4778-842d-bd7ba01fc686 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:21:48.588913 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:21:48.589656 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:21:48.590546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:21:48.590758 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:21:48.600150 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:21:48.601597 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:21:48.611160 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1475/2919] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:21:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:21:58.606675 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=15 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:22:01.363161 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] REQUEST_METHOD: `POST` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:01.363869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:01.363869 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:01.382542 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.auth.core [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] MFA Rules not processed for user `d13d9b9d3e48455082a74eaa3f1b4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=92305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:22:01.383796 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:22:01.391073 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1445/2920] 10.222.0.74 () {58 vars in 957 bytes} [Wed May 25 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:22:01.432015 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:01.441741 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2efa79-f74b-4292-8a4a-197ee1b95e62 service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:01.445789 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea1035a1d2434e948256087a112b4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:01.446435 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:01.446754 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:01.447037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:01.466829 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:01.468298 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:01.492479 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1476/2921] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:01.803263 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:01.817067 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455f7d3-b05f-4075-98a8-1417e0d1029d None None] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:01.817622 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:01.818612 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:01.818937 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:01.819219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:01.827988 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:01.829501 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:01.840071 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1446/2922] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:02.661091 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:02.672489 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d07496-faa6-4ef2-83f7-aab97a3c50cc service cinder] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:02.678050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:02.678926 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:02.679218 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:02.679502 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:02.693409 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:02.694197 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:02.708473 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1477/2923] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:05.985295 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:06.000724 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cc145-0d3b-4679-9790-4328b07b5caa service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:06.007546 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:06.008717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:06.009172 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:06.009688 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:06.024675 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:06.028259 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:06.046852 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1447/2924] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:07.595546 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:07.610515 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd82b3-05ce-42c5-a942-042c4125a96a service neutron] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:07.617318 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4beeef3c39c44f39c8c7ee5f13e9623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:07.622720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:07.622720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:07.622720 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:07.632483 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:07.634499 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:07.647874 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1478/2925] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:22:16.036785 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=15 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:22:17.643286 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=25 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:22:23.158925 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:23.169383 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6167ad-fc27-4fa6-8830-ff6fcef1d7c4 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:23.173510 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:23.174185 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:23.174474 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:23.174802 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:23.187004 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:23.188478 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:23.199953 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1448/2926] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:33.194567 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=6 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:54.826097 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:54.840834 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47cb7-461a-46eb-927c-d5ac3c77de88 service nova] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:54.845901 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:54.846556 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:54.846883 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:54.847163 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:54.856396 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:54.858036 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:54.868621 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1479/2927] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:04.863953 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=6 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:41.307275 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.316123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ced4b-3441-4ca7-87fc-28b4111c4c7e service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.319800 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '931dba704d244ccdbb9f7dbadc68d7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.320940 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.321219 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.321494 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.396187 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.397812 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.421635 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1449/2928] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:44.917224 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:44.918975 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:44.934004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f7835-c2d4-481b-bdf2-f9c3bfc51898 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:44.941384 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:44.942004 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:44.942272 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:44.942549 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:44.944437 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:44.958232 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:44.960037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:44.971914 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1480/2929] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:24:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:48.623427 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:48.634935 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2d849-55f1-4bbc-bc74-1646cfdfa033 service neutron] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:48.639916 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:48.640560 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] REQUEST_METHOD: `GET` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:48.640844 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:48.641123 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:48.643015 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:48.655954 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:48.657571 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:48.668448 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1450/2930] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:48.759968 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:48.770750 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d659-1a99-4052-93ff-ba481c83fcf3 service placement] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:48.775428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:48.776303 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:48.776589 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:48.776872 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] PATH_INFO: `/v3/users/d13d9b9d3e48455082a74eaa3f1b4f18` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:48.778081 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13d9b9d3e48455082a74eaa3f1b4f18)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:48.780136 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:48.966043 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13d9b9d3e48455082a74eaa3f1b4f18'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:48.973382 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13d9b9d3e48455082a74eaa3f1b4f18'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:48.974045 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13d9b9d3e48455082a74eaa3f1b4f18'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:48.986615 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13d9b9d3e48455082a74eaa3f1b4f18'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.048131 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13d9b9d3e48455082a74eaa3f1b4f18'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.054699 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1481/2931] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:24:48 2022] DELETE /identity/v3/users/d13d9b9d3e48455082a74eaa3f1b4f18 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:49.369124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:49.378100 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fefa8-a626-4ab6-92ea-03e74ffb3c82 service placement] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:49.440574 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:49.441465 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:49.441751 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:49.442030 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] PATH_INFO: `/v3/projects/9e5a54632992428ca774d47f3f20d6b9` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:49.443254 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5a54632992428ca774d47f3f20d6b9)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:49.445792 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:49.553718 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5a54632992428ca774d47f3f20d6b9', 'request_id': 'req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.634426 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5a54632992428ca774d47f3f20d6b9', 'request_id': 'req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.641335 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5a54632992428ca774d47f3f20d6b9', 'request_id': 'req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.645514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:49.653225 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] The token cache is being invalidate because project 9e5a54632992428ca774d47f3f20d6b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:49.653513 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.654149 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.655744 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1451/2932] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:24:49 2022] DELETE /identity/v3/projects/9e5a54632992428ca774d47f3f20d6b9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:49.659833 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:49.661253 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:49.674375 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104304ec-43fd-4a7a-842c-f1917fa2454b admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:49.740907 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:49.742215 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:49.742507 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:49.742815 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] PATH_INFO: `/v3/users/f834b244b18840989978b168d6dd28f2` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:49.744049 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f834b244b18840989978b168d6dd28f2)` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:49.745997 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:49.978616 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f834b244b18840989978b168d6dd28f2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.987979 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f834b244b18840989978b168d6dd28f2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.988778 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f834b244b18840989978b168d6dd28f2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:49.994184 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:50.002899 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f834b244b18840989978b168d6dd28f2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.125387 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.notifications [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f834b244b18840989978b168d6dd28f2'} {{(pid=92306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.132441 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1482/2933] 10.222.0.74 () {58 vars in 1223 bytes} [Wed May 25 15:24:49 2022] DELETE /identity/v3/users/f834b244b18840989978b168d6dd28f2 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:50.378336 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Authenticating user token {{(pid=92305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:50.380124 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.fernet_utils [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:50.394858 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb51b8c-1110-42f5-a4d3-90da6e0f3409 admin admin] Validating token access rules against request {{(pid=92305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:50.461121 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e695952cd3d8473291efec1acc9a5e16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77e7e794d9844bf98e02bc9b4f6dae9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:50.461945 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:50.462233 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:50.462547 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] PATH_INFO: `/v3/projects/3b45574471b34bd9be7ec77f34c48f75` {{(pid=92305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:50.463799 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b45574471b34bd9be7ec77f34c48f75)` {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:50.465768 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] RBAC: Authorization granted {{(pid=92305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:50.657717 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b45574471b34bd9be7ec77f34c48f75', 'request_id': 'req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.771062 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b45574471b34bd9be7ec77f34c48f75', 'request_id': 'req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.778002 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b45574471b34bd9be7ec77f34c48f75', 'request_id': 'req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2'} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.788691 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] The token cache is being invalidate because project 3b45574471b34bd9be7ec77f34c48f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:50.788984 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.789643 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG keystone.notifications [None req-7d08fb6e-82cd-4d96-bb26-b69aca5f42a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:50.791218 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: [pid: 92305|app: 0|req: 1452/2934] 10.222.0.74 () {58 vars in 1232 bytes} [Wed May 25 15:24:50 2022] DELETE /identity/v3/projects/3b45574471b34bd9be7ec77f34c48f75 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:00.262219 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:00.785514 dv-cinder-830283-3-iscsi devstack@keystone.service[92305]: DEBUG dbcounter [-] [92305] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=92305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:29:00.816037 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Authenticating user token {{(pid=92306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:29:00.817428 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:29:00.958557 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d98c8-116d-432d-aec4-6564cfe081fd admin admin] Validating token access rules against request {{(pid=92306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:29:01.017567 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd162deab164e70b629f8c88495ccb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c07159d21300478e88a07006614fdded', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:29:01.018225 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] REQUEST_METHOD: `GET` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:29:01.018514 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] SCRIPT_NAME: `/identity` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:29:01.018826 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:29:01.020639 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.fernet_utils [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:29:01.032460 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:29:01.034050 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba27780-8a30-4be2-bea8-31c9a80a3076 service placement] RBAC: Authorization granted {{(pid=92306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:29:01.043928 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: [pid: 92306|app: 0|req: 1483/2935] 10.222.0.74 () {60 vars in 1365 bytes} [Wed May 25 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:29:11.039980 dv-cinder-830283-3-iscsi devstack@keystone.service[92306]: DEBUG dbcounter [-] [92306] Writing DB stats keystone:SELECT=89 {{(pid=92306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}